What will be an efficient way of converting a std::vector to a std::vector

Question

I can use a loop to copy elements from boost::any vector to std::vector of floats.

It is inefficient in the sense that there are 50000 values to copy, vector can grow beyond 50K and I have to save the to the disk in different folders. So it is a bottleneck.

So far the following wont work

vector<boost::any> v1
vector<float> v2

std::fill(v1.begin().v1.end(),v2) // This will not copy anything.

Can you explain what you mean by inefficient? If you want to turn a 50000-element any-vector into a 50000-element float-vector, you're going to have to copy 50000 elements. — Sebastian Redl, May 25 '21 at 11:06
The actual problem is that I have to do this repeatedly and save the vector to legacy std::vectors due do design constraints. Some vectors are beyond 100K or even more. So the loop approach is basically a bottleneck at them moment. — Wajih, May 25 '21 at 11:10
Do you really need that vector of `any`? If you know that floats are stored inside, why to use `any`? — Daniel Langr, May 25 '21 at 11:17
@DanielLangr - Again I have to stick with what is available. Last resort would be to ditch boost::any all together. — Wajih, May 25 '21 at 11:21
Ditching boost::any as you mention would be a better design. That class is meant for cases where multiple types must be handled together in a single container. If all contents are of the same type it only adds overhead and complexity without gaining anything. — olm, May 25 '21 at 11:54
@olm if you have e.g. a JSON document, you might find that you have something like a `ns::vector` that you know contains only `float`s — Caleth, May 25 '21 at 11:56

Leontyev Georgiy · Answer 1 · 2021-05-25T11:23:02.017

2

According to this,

vector<boost::any> v1;
vector<float> v2;

v2.reserve(v1.size());
std::transform(std::begin(v1), std::end(v1), std::back_inserter(v2), 
[] (const auto& v1_val) { return boost::any_cast<float>(v1_val); });

will work.

... reserve(...) is the most important line here [ so that you don't reallocate memory when you add more and more items to the vector ].

To be honest, I don't really understand how to do it efficiently (memcpy-like), because you have a number of objects of different type, and to add it to another container, you have to cast EACH one to that type. You can't possibly do it more efficient than with a simple cycle or what other STL stuff provides (which is also a simple cycle anyway).

edited May 25 '21 at 11:23

answered May 25 '21 at 11:09

Leontyev Georgiy

1,295
11
24

Let me give this a try sir! – Wajih May 25 '21 at 11:11
To be honest, I didn't run that. Probably, there will be a problem with a cast from Boost::any to float. In that case, within lambda, you should write a code that converts a single Boost::any (v1_val) to a float (what is returned). – Leontyev Georgiy May 25 '21 at 11:13
2

@Wajih std::transform IS a loop anyway, man. – Leontyev Georgiy May 25 '21 at 11:13
seems that the implicit cast wont work. I get an error at compile time "[no known conversion for argument 1 from ‘boost::any’ to ‘const float&’ " – Wajih May 25 '21 at 11:18
Sad that there is standard way from boost of such conversions – Wajih May 25 '21 at 11:19
@Wajih `any` does not know what is stored inside due to _type erasure_. Consequently, implicit cast can't work. You need to cast explicitly. (At least with `std::any`. I am not that much familiar with `boost::any`, but I guess they are the same in this aspect.) – Daniel Langr May 25 '21 at 11:20
return boost::any_cast(v1_val); should work – Leontyev Georgiy May 25 '21 at 11:21
@Wajih Answer edited to apply fixes and a link to reference. – Leontyev Georgiy May 25 '21 at 11:23
@LeontyevGeorgiy - I expected it to work, but still get almost the same compile time error "no known conversion for argument 1 from ‘boost::any’ to ‘const int&’" – Wajih May 25 '21 at 11:27
Should there be a way using std::vector::data pointer? – Wajih May 25 '21 at 11:29
const int? We were working with float, af far as I thought... – Leontyev Georgiy May 25 '21 at 11:31
The only way of using it is to roll through its data pointer with a cycle and convert all items manually. – Leontyev Georgiy May 25 '21 at 11:31
BTW, this is hardly an answer, since its solution is based on copying, which OP wants to avoid. – Daniel Langr May 25 '21 at 11:37
I don't think you need to wrap `boost::any_cast` in a lambda, it's a callable already – Caleth May 25 '21 at 11:54
@LeontyevGeorgiy yes we are working with floats. I dont understand why it gives me a compile error for const int& – Wajih May 25 '21 at 12:49

Daniel Langr · Answer 2 · 2021-05-25T12:26:38.027

Your problem cannot be solved without copying. When you store some floats into a std::vector<boost::any>, those floats will be stored in memory at completely unrelated places. This is caused by the applied type eraseure technique.

Live demo: https://godbolt.org/z/a91qWazK7

You can observe that those floats have different addresses than are the addresses of any objects inside a vector. This means that the floats are not stored in the vector's buffer itself. (In theory, they could be stored in this buffer if some small buffer optimization technique would be applied by boost::any, which doesn't seem to happen. However, even then, this wouldn't help since the elements wouldn't be store contiguously in memory.)

Bottom line: If you need to get floats stored in std::vector<boost::any> to std::vector<float>, there is no way to do that without copying them.

What will be an efficient way of converting a std::vector to a std::vector

2 Answers2