Generalising multiple nested for loops

Question

I have a data structure like this: map<string, map<string, map<string, MyObj>>>

Now, I have multiple functions all of which use the same for loop method:

for (auto p1 : myMap)
    for (auto p2 : p1.second)
        for (auto p3 : p2.second)
            doThingsWith(p1, p2, 3);

Where the doThingsWith(p1, p2, p3) varies between functions, as well as the code before and after the for loops. Also, some functions only need to access MyObj objects for example, while others need access to all the string keys as well as the MyObj objects.

So, the question is, is there any method to generalise this without losing performance? I came up with a function that returns a vector of tuples:

vector<tuple<string, string, string, MyObj>> getData(... &myMap)
{
    vector<tuple<string, string, string, MyObj>> data;
    for (auto p1 : myMap)
        for (auto p2 : p1.second)
            for (auto p3 : p2.second)
                data.push_back(tuple<string, string, string, MyObj>(
                    p1.first, p2.first, p3.first, p3.second
                ));
    return data;
}

And now my functions can use this:

for (auto t : getData(myMap))
    doThingsWith(get<0>(t), get<1>(t), get<2>(t), get<3>(t));

But this is unnecessarily constructs a lot of tuples and vectors, since myMap is huge.

Is there any better way? In Python I can use generators, but I don't know the C++ equilevant:

def iterTuples(myMap):
    for k1, v1 in myMap.items():
        for k2, v2 in v1.items():
            for k3, v3 in v2.items():
                yield k1, k2, k3, v3


for k1, k2, k3, val in iterTuples(myMap):
    doThingsWith(k1, k2, k3, val)

You can define iterator in the same way as Python's generator. — llllllllll, Feb 21 '18 at 12:12
Maybe try storing `reference_wrapper`s inside tuples? This will avoid copying, but still alow You to get the data inside the tuples — bartop, Feb 21 '18 at 12:23

John Zwinck · Accepted Answer · 2018-02-21T12:52:58.787

3

Your map-of-maps-of-maps is from the beginning not efficient. Better would be to consolidate the key parts:

typedef tuple<string, string, string> Key;
std::map<Key, MyObj> myMap;

Now you have a straightforward efficient solution:

for (const auto& pr : myMap)
    doThingsWith(get<0>(pr.first), get<1>(pr.first), get<2>(pr.first), pr.second);

And you avoid a lot of indirection. Even more if you can use fixed-length strings and/or combine the strings into a single allocation. Or if it's important to share the first and second key strings, you can use a reference-counted string type.

edited Feb 21 '18 at 12:52

answered Feb 21 '18 at 12:29

John Zwinck

239,568
38
324
436

1

*"Your map-of-maps-of-maps is from the beginning not efficient."*. Depends if OP has usage with intermediate maps. – Jarod42 Feb 21 '18 at 15:40

score 0 · Answer 2 · answered Feb 21 '18 at 12:28

Simply create template functions:

template <typename Map3, typename F>
void ForEachAll(Map3&& m, F&& f)
{
    for (auto&& p1 : m)
        for (auto&& p2 : p1.second)
            for (auto&& p3 : p2.second)
                f(p1.first, p2.first, p3.first, p3.second);
}

template <typename Map3, typename F>
void ForEach(Map3&& m, F&& f)
{
    for (auto&& p1 : m)
        for (auto&& p2 : p1.second)
            for (auto&& p3 : p2.second)
                f(p3.second);
}

Note:

You can name them identically with some SFINAE
MAP3 is template to allow const and no const map.

Generalising multiple nested for loops

2 Answers2