3

For example, why doesn't template< typename Elem, typename Traits, typename Alloc > basic_string { ... } provide:

template< typename OtherAlloc >
basic_string( const basic_string< Elem, Traits, OtherAlloc >& a_Other ) { ... }

It would seem fairly trivial to implement such a conversion constructor which respects both allocators. The current state of affair makes it very cumbersome to interface between types just differing in allocators.

Mooing Duck
  • 64,318
  • 19
  • 100
  • 158
Ylisar
  • 4,293
  • 21
  • 27
  • 1
    It probably wouldn't be too hard to implement a template function that converts between std template types that differ only in their allocator. (I'm not saying this question isn't valid -- it certainly is -- only that the amount of code required to work around this shouldn't be much.) – cdhowie Jun 11 '12 at 20:31
  • Yeah, as conversion can't be a free operator I would have to explicitly convert throughout my code base however, still might be the only solution however :( – Ylisar Jun 11 '12 at 20:54
  • Yes, automatic conversion won't be possible unfortunately. – cdhowie Jun 11 '12 at 21:02
  • My custom containers don't support different allocators, but that was because I thought there would be a conflict when working with containers with different `pointer` types or something. I can't think of how that would hinder anything now, as long as they have the same `value_type`. – Mooing Duck Jun 11 '12 at 21:07

2 Answers2

3

The Standard hash also does not permit allocator fun- it can hash std::string and std::wstring but not std::basic_string<char, std::char_traits<char>, custom_alloc>. In addition, you cannot create an unordered_map or unordered_set with just an allocator- you must also provide a bucket number, which defaults to an implementation-defined constant you can't access, so you effectively must just make something up. The support, generally, is not very good.

It seems to me that, relatively simply, nobody proposed such a function or explored this use space.

Puppy
  • 144,682
  • 38
  • 256
  • 465
  • It's hard to believe this didn't come up when working with `tuple` and `pair` and such, there _has_ to be a reason for this. :( – Mooing Duck Jun 11 '12 at 21:08
  • @MooingDuck: What do `tuple` and `pair` have to do with allocators? And many omissions in the standard happen because nobody proposed them; why would this be any different? – Nicol Bolas Jun 11 '12 at 21:47
  • @NicolBolas: You can construct `tuple` and `pair` from a `tuple`/`pair` containing different types _including a different allocator type_, and it works _almost exactly_ in the way that the OP is asking for. If `tuple`/`pair` nearly do it, why not the "real" containers? (C++11 Jan2012 §20.4.2.1/26) – Mooing Duck Jun 11 '12 at 21:51
  • @MooingDuck: Because `tuple` doesn't have the allocator as part of their *type*. The containers do, and there's no way to change that since it's a part of C++98 without breaking a lot of code. `tuple` was adopted from Boost, where it stored the allocator automagically, much like `shared_ptr` and other Boost types. They can't simply decide to change how allocators work on already existing types. Also, `pair` doesn't have allocator constructors. – Nicol Bolas Jun 11 '12 at 22:07
  • @NicolBolas: I still don't see the problem though. The constructor would basically just be `assign(rhs.begin(), rhs.end());` and that's it. I don't see how "the allocator is different" is related to anything. – Mooing Duck Jun 11 '12 at 22:10
  • @MooingDuck: As I said, the reason why `tuple` has it is because it came from *Boost*. The standards committee inherited it. I'm not saying it's impossible to add the constructors and assignment operators. I'm saying that it was an *oversight* that they're not there, and the fact that `tuple` has them doesn't mean it isn't still an oversight. – Nicol Bolas Jun 11 '12 at 22:12
1

The problem is much harder than it looks. Because the allocator type is part of the type of the object there is little interaction allowed between types that differ only in allocator. The first example that comes to mind is that a function that takes a std::string by constant reference cannot take a string that uses a different allocator. One particular such case is during construction of objects. As a matter of fact it is probably the harder case here.

Consider for example the following code:

// Assume that you could construct from a different allocator
std::vector<int, allocator1> f() {
   std::vector<int, allocator2> r;
   // fill in
   return r;
}
int main() {
   std::vector<int, allocator3> x = f();
}

Consider that allocator1 is std::allocator<int> (i.e. the default allocator), that allocator2 uses a local arena in the stack, and that allocator3 might use shared memory. In theory the code is quite simple, the vector r is created and filled with data, on the return statement a new temporary is created by copying from r, and finally x is constructed by copying from that temporary. The problem is that the standard allows (and compilers like) to avoid copying as much as possible. In the particular example above (and ignoring allocators) the compiler would elide both copies and create the buffer only once, which is fast and efficient. But with allocators potentially being different, NRVO and other types of copy-elision would have to be disabled. (If those optimizations are enabled, x in main would be using allocator2, with a local arena that has already been destroyed, causing undefined behavior).

By enabling copy construction from containers with one allocator into another you might end up in a mess, or in a deeper mess than we are already in the current standard, where you can cause all sort of interesting issues with stateful allocators (say that you use per-thread allocators, and that you move data into a shared queue, you might end up with one thread holding an object created by the per-thread allocator on another thread, and because the point of using per-thread allocators is avoid contentions on locks, you could be creating a race condition on apparently safe code....


This is an old proposal to the C++ committee Towards a better allocation model that raises some of the concerns with the C++03 allocation model and proposes a polymorphic allocator type (which has problems of it's own). It makes for an interesting read, but beware of the details, not everything is as good as it seems, and there are quite a few pitfalls in using either option (or the C++11 version that is similar to the C++03 version)

David Rodríguez - dribeas
  • 204,818
  • 23
  • 294
  • 489
  • 1
    Yupp, no NRVO in the case where allocators differ. I understand that any sort of moving of data is strictly prohibited when allocators doesn't match, but it would seem to me that it would be fairly easy to take advantage of the best case when allocators match due to the best specialization rules. – Ylisar Jun 11 '12 at 21:02
  • @Ylisar: The problem is that the compiler cannot possibly know whether the allocators match in the second copy elision. That is, the caller allocates space for the object in it's context and then calls the function. The function can elide the copy from `r` to the return statement, but it cannot know whether the allocator in the context of the caller is the same or compatible. Also note that this is harder to track than what it looks. How do you know (without actual knowledge of the allocator implementation) whether the allocator can be reused or not? – David Rodríguez - dribeas Jun 11 '12 at 21:16
  • @MooingDuck: You have missed that up to the last paragraph is hipothetical assuming as a premise that copy construction from a different allocator type was allowed. That is, if what the Q. asks about was allowed you would run into the issues above, which means that either you disable all copy elision for all types that have an allocator, or you are treading in the razor's edge... – David Rodríguez - dribeas Jun 11 '12 at 21:16
  • 1
    @DavidRodríguez-dribeas: Alright, as you say, such a constructor could not be a _copy constructor_ as that would make no sense. However, the OP is asking about an additional _conversion constructor_ which would not have any of the problems you mentioned. And the problems in the last paragraph apply to similar code even without the conversion constructor, so I fail to see how it's relevant. (It might be that it _is_ relevant, but I don't see it after two reads) – Mooing Duck Jun 11 '12 at 21:24
  • @MooingDuck: It is relevant to make the point that playing with allocators is much more problematic than what looks on the surface. Including the fact that (IIRC) none of the containers in C++11 complies with the *swappable* requirement in the standard specifically because in C++11 allocators have state, which is a much smaller step than mixing different allocator types. The problem is that allocators live in an orthogonal space, and many of the magic that compilers do cannot be done with allocators. The allocator model in the standard is broken, and the alternatives are not much better... – David Rodríguez - dribeas Jun 11 '12 at 21:32
  • ... more efficient, probably, but dangerous in the same ways. The easier it is to mix allocators, the easier it is to cause all sorts of issues that cannot be detected by the compiler. While in C++03 allocators were too limited to be useful, in C++11 they are a bit more powerful and a lot more dangerous. An any extra feature you add that enables mix and match, provides greater power and danger to the mix. I have spent some time discussing allocators with some people, including John Lakos (the author of the allocator library that was proposed for the standard), and I don't like any... – David Rodríguez - dribeas Jun 11 '12 at 21:34
  • ... of the alternatives. The different options are different compromises of efficiency or fragility of code. I want my code efficient and I want the compiler to help me detecting issues, and that combination seems unattainable. – David Rodríguez - dribeas Jun 11 '12 at 21:36
  • 2
    @DavidRodríguez-dribeas: I don't know what you mean by the containers not meeting the swappable requirement, as the allocators are also swapped, and nobody is proposing being able to swap containers with different allocators. None of this explains why we can't have a conversion constructor that simply makes a copy of the data, even if the allocator model is broken. – Mooing Duck Jun 11 '12 at 21:39
  • 1
    @Ylisar: BTW, there is a simple workaround for your problem: construct using iterators. There is no problem in constructing from completely different containers by forcing copies through the extra level of indirection. But trying to make the operation a bit more specialized only adds chances of code not doing what you expect in ways that are horrible to debug (debugging issues with memory allocation is one of the hard problems, as memory changes in ways that are not directly shown in the code. Been there, done that.) – David Rodríguez - dribeas Jun 11 '12 at 21:39
  • @MooingDuck: I need to get back and find what was the corner case not supported by the standard... At any rate, a *value conversion* is attainable through the constructor that takes iterators, and anything else (that is, anything that does not explicitly *force* making copies) would be a disaster waiting to happen. The issue with the standard is just an indication of the complexity of the problem. The people in the committee are smart people, and some of the interactions of allocators with containers got messed up. That is an indication of the complexity of each new operation on allocators. – David Rodríguez - dribeas Jun 11 '12 at 22:21
  • @DavidRodríguez-dribeas: I agree that anything besides a forced value conversion would be a disaster. I also agree that allocators have unexpected holes, and would be interested to see any corner-case I might be overlooking. – Mooing Duck Jun 11 '12 at 22:30
  • The standard requires that the elements in the range passed to many algorithms (including sort) are `ValueSwappable`. `ValueSwappable` requires that any two objects of the type obtained by dereferencing the iterator are *swappable with* the other. For any allocator that does not propagate on copy/swap, that means that even if all of the objects in the container are allocated with the exactly the same allocator, you cannot (according to the standard) `std::sort` the contents. – David Rodríguez - dribeas Jun 12 '12 at 00:09
  • As of corner cases, any case in which you have a stateful allocator and the container (include variants of `std::string`) can be *moved* or *swapped* as the allocator will be moved along with the data silently (consider a queue that binds two threads, one thread reading data from the network from packets of fixed length (twitter) that are allocated through an efficient per-thread fixed-size arena. If the strings are moved to the queue, or if they are moved out of the queue by the consumer thread the thread that allocated an the thread that deallocated will be different, potentially causing UB. – David Rodríguez - dribeas Jun 12 '12 at 00:13
  • Consider a local function that holds a `std::set`, and for efficiency uses a stack bound arena allocator. If the `std::set` is moved out of the function the container will outlive the allocator causing UB. Basically *any* allocator con state is prone to *optimizations* that end up causing UB. We have our own standard library implementation that implements the allocator model in the proposal above, and I am quite aware of the extra awareness you need to have to manage the extra performance. – David Rodríguez - dribeas Jun 12 '12 at 00:22
  • But the allocator is part of the type throughout std, so I don't see how any of your examples can possibly happen. The only time I can see this possibly occurring is if an object were to hold an allocator by ref / ptr ( unless I'm missing something ). The solution is to not implement `swap` et al as a memory area swap when the allocators doesn't match, something which is easy to do while maintaining all the current benefits. – Ylisar Jun 12 '12 at 04:38
  • 1
    @Ylisar: Part of the changes in C++ allow for stateful allocators. That means that not all instances of the same allocator type are the same (that used to be true in C++03, where the standard required that all allocators from a given type *were* equal). And that brings some of the problems above. The n1850 proposes the use of polymorphic allocators, which means that the container can be parametrized with a base type but hold different allocators internally --this solves your issues as the types are the same, but brings other potential pitfalls into play. – David Rodríguez - dribeas Jun 12 '12 at 12:38