multi_index container ordered_non_unique is not rearranging the order after updating the key

Question

I have created multi_index container element and updating one of the ordered non unique key without calling modify or update. Will this container rearrange based on the new update value?

Sample program:

#include <boost/multi_index_container.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/hashed_index.hpp>
#include <boost/multi_index/sequenced_index.hpp>
#include <boost/multi_index/random_access_index.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/global_fun.hpp>
#include <boost/multi_index/mem_fun.hpp>
#include<boost/shared_ptr.hpp>

using namespace boost::multi_index;
using namespace boost::multi_index::detail;
using namespace boost;

class Element {
public:
    int id;
    int time;
    std:: string name;
    Element (int id_, int time_)
    {
         id = id_;
         time = time_;
         name = "";
    }
    int get_id() { return id;}
    int get_time() { return time;}
    std ::string get_name(){return name;}
};


typedef multi_index_container <

            boost::shared_ptr <Element>,

            indexed_by <


                hashed_unique <
                    BOOST_MULTI_INDEX_MEM_FUN (Element, int, get_id)
                >,


                ordered_non_unique <
                    BOOST_MULTI_INDEX_MEM_FUN (Element, int, get_time)
                >
            >
        > ElementDB;

using namespace boost::multi_index;
using namespace std;

ElementDB mElementDB;
int main()
{
    ElementDB mElementDB;
    boost::shared_ptr <Element> ElementInfo;
    typedef nth_index <ElementDB, 0>::type id_map;
    typedef id_map::iterator id_iterator_t;

    typedef nth_index <ElementDB, 1>::type time_map;
    typedef time_map::iterator time_iterator_t;

    ElementInfo = boost::shared_ptr <Element> (new Element (1, 10));
    ElementInfo -> name = "abc";
    mElementDB.insert(ElementInfo);
    ElementInfo = boost::shared_ptr <Element> (new Element (2, 11));
    ElementInfo -> name = "def";
    mElementDB.insert(ElementInfo);
    ElementInfo = boost::shared_ptr <Element> (new Element (3,12));
    ElementInfo -> name = "ghi";
    mElementDB.insert(ElementInfo);

    printf("Size of Element Map:%zu\n", mElementDB.size());
    time_iterator_t start = mElementDB . get <1> () . begin ();
    time_iterator_t end = mElementDB . get <1> () . end ();
    time_iterator_t time_iter = start;
    while (start != end)
    {
        ElementInfo = *time_iter;
        if(ElementInfo)
        {
                printf("id: %d time:%d name:%s\n", ElementInfo -> get_id(),ElementInfo -> get_time(), ElementInfo-> get_name(). c_str());
        time_iter++;
        }
    }

    id_iterator_t iter = mElementDB.get <0> ().find (2);
    if (iter != mElementDB.get <0> ().end())
    {
        printf("id: %d time:%d\n", (*iter) -> get_id(), (*iter) -> get_time());
        (*iter) -> time = 5;
        (*iter) -> name = "xyz";
    }
    else
        printf("No Element found with id");

    start = mElementDB . get <1> () . begin ();
    end = mElementDB . get <1> () . end ();
    time_iter = start;
    while (time_iter != end)
    {
        ElementInfo = *time_iter;
        printf("id: %d time:%d name:%s\n", ElementInfo -> get_id(), ElementInfo -> get_time(),ElementInfo->get_name().c_str() );
        mElementDB.get <1> () . erase (time_iter);
        start = mElementDB . get <1> () . begin ();
        end = mElementDB . get <1> () . end ();
        time_iter = start;
    }
    return 0;
}

My application is doing the similar update like above on the key and non-key update without using modify or replace. Is the modify or replace needed only for the keys? Can we update non-keys without modify or replace? I am also using few composite keys in the map but here I only showed two keys.

I am seeing crash application crash at the when inserting and erasing at random times. Are these crashes happening as I am updating the key without modify or replace? I am only updating only one key that is ordered non unique and all other keys are hashed unique and hashed non unique. I am using boost version 1.58.0 and this is the crash back trace when erasing element from the map. Is there anything we can point out what is corrupting from the back trace?

#4  operator()<boost::multi_index::detail::hashed_index_node_impl<std::allocator<char> >*> (this=<optimized out>, val=<synthetic pointer>, x=@0x0: <error reading variable>)
    at /usr/include/boost/multi_index/detail/hash_index_node.hpp:167
#5  left_unlink_last_of_group<boost::multi_index::detail::default_assigner> (assign=..., x=0x7f247a1345f8) at /usr/include/boost/multi_index/detail/hash_index_node.hpp:651
#6  boost::multi_index::detail::hashed_index_node_alg<boost::multi_index::detail::hashed_index_node_impl<std::allocator<char> >, boost::multi_index::detail::hashed_non_unique_tag>::unlink<boost::multi_index::detail::default_assigner> (
    x=0x7f247a1345f8, assign=...) at /usr/include/boost/multi_index/detail/hash_index_node.hpp:494
#7  0x00007f24b8fd58c5 in unlink (x=0x7f247a1345f8) at /usr/include/boost/multi_index/detail/hash_index_node.hpp:444
#8  unlink (this=<optimized out>, x=0x7f247a1345d0) at /usr/include/boost/multi_index/hashed_index.hpp:1262
#9  erase_ (x=0x7f247a1345d0, this=0x7f24b91f2690 <ElementDB::mElementDb+16>) at /usr/include/boost/multi_index/hashed_index.hpp:814
#10 erase_ (x=0x7f247a1345d0, this=0x7f24b91f2690 <ElementDB::mElementDb+16>) at /usr/include/boost/multi_index/hashed_index.hpp:815
#11 erase_ (x=0x7f247a1345d0, this=0x7f24b91f2690 <ElementDB::mElementDb+16>) at /usr/include/boost/multi_index/hashed_index.hpp:815
#12 erase_ (x=0x7f247a1345d0, this=0x7f24b91f2690 <ElementDB::mElementDb+16>) at /usr/include/boost/multi_index/hashed_index.hpp:815
#13 erase_ (x=0x7f247a1345d0, this=0x7f24b91f2680 <ElementDB::mElementDb>) at /usr/include/boost/multi_index_container.hpp:760
#14 final_erase_ (x=0x7f247a1345d0, this=0x7f24b91f2690 <sElementDB::mElementDb+16>) at /usr/include/boost/multi_index/detail/index_base.hpp:259
#15 erase (position=..., this=0x7f24b91f2690 <sElementDB::mElementDb+16>) at /usr/include/boost/multi_index/hashed_index.hpp:311

Please describe the context of your question and what you tired so far, provide a [mcve]! — hc_dev, Nov 19 '19 at 23:30
Thank you for your comment @hc_dev. I have added some more information. — Ansh, Nov 21 '19 at 00:20

Joaquín M López Muñoz · Answer 1 · 2019-11-21T10:56:15.073

The answer is no. Modifying a key by means other than using modify or modify_key will likely lead to a crash or erratic behavior.

As for your complete sample code, I've detected some problems:

1: Line 78

while (start != end)

should be

while (time_iter != end)

2: The following, as discussed above, is incorrect:

(*iter) -> time = 5;
(*iter) -> name = "xyz";

Instead, use modify:

mElementDB.get <0> ().modify(iter, [](auto& p){
    p -> time = 5;
    p -> name = "xyz";
});

3: The final erasing loop is unnecessarily complex, you can simply write:

while (time_iter != end)
{
    ElementInfo = *time_iter;
    printf("id: %d time:%d name:%s\n", ElementInfo -> get_id(), ElementInfo -> get_time(),ElementInfo->get_name().c_str() );
    time_iter = mElementDB.get <1> () . erase (time_iter);
}

Complete retouched sample follows:

Live On Wandbox

#include <boost/multi_index_container.hpp>
#include <boost/multi_index/ordered_index.hpp>
#include <boost/multi_index/hashed_index.hpp>
#include <boost/multi_index/sequenced_index.hpp>
#include <boost/multi_index/random_access_index.hpp>
#include <boost/multi_index/member.hpp>
#include <boost/multi_index/global_fun.hpp>
#include <boost/multi_index/mem_fun.hpp>
#include<boost/shared_ptr.hpp>

using namespace boost::multi_index;
using namespace boost::multi_index::detail;
using namespace boost;

class Element {
public:
    int id;
    int time;
    std:: string name;
    Element (int id_, int time_)
    {
         id = id_;
         time = time_;
         name = "";
    }
    int get_id() { return id;}
    int get_time() { return time;}
    std ::string get_name(){return name;}
};


typedef multi_index_container <

            boost::shared_ptr <Element>,

            indexed_by <


                hashed_unique <
                    BOOST_MULTI_INDEX_MEM_FUN (Element, int, get_id)
                >,


                ordered_non_unique <
                    BOOST_MULTI_INDEX_MEM_FUN (Element, int, get_time)
                >
            >
        > ElementDB;

using namespace boost::multi_index;
using namespace std;

ElementDB mElementDB;
int main()
{
    ElementDB mElementDB;
    boost::shared_ptr <Element> ElementInfo;
    typedef nth_index <ElementDB, 0>::type id_map;
    typedef id_map::iterator id_iterator_t;

    typedef nth_index <ElementDB, 1>::type time_map;
    typedef time_map::iterator time_iterator_t;

    ElementInfo = boost::shared_ptr <Element> (new Element (1, 10));
    ElementInfo -> name = "abc";
    mElementDB.insert(ElementInfo);
    ElementInfo = boost::shared_ptr <Element> (new Element (2, 11));
    ElementInfo -> name = "def";
    mElementDB.insert(ElementInfo);
    ElementInfo = boost::shared_ptr <Element> (new Element (3,12));
    ElementInfo -> name = "ghi";
    mElementDB.insert(ElementInfo);

    printf("Size of Element Map:%zu\n", mElementDB.size());
    time_iterator_t start = mElementDB . get <1> () . begin ();
    time_iterator_t end = mElementDB . get <1> () . end ();
    time_iterator_t time_iter = start;
    while (time_iter != end)
    {
        ElementInfo = *time_iter;
        if(ElementInfo)
        {
                printf("id: %d time:%d name:%s\n", ElementInfo -> get_id(),ElementInfo -> get_time(), ElementInfo-> get_name(). c_str());
        time_iter++;
        }
    }

    id_iterator_t iter = mElementDB.get <0> ().find (2);
    if (iter != mElementDB.get <0> ().end())
    {
        printf("id: %d time:%d\n", (*iter) -> get_id(), (*iter) -> get_time());
        mElementDB.get <0> ().modify(iter, [](auto& p){
            p -> time = 5;
            p -> name = "xyz";
        });
    }
    else
        printf("No Element found with id");

    start = mElementDB . get <1> () . begin ();
    end = mElementDB . get <1> () . end ();
    time_iter = start;
    while (time_iter != end)
    {
        ElementInfo = *time_iter;
        printf("id: %d time:%d name:%s\n", ElementInfo -> get_id(), ElementInfo -> get_time(),ElementInfo->get_name().c_str() );
        time_iter = mElementDB.get <1> () . erase (time_iter);
    }
    return 0;
}

Regarding your question of whether non-keys can be modified without modify, in principle you can, though my advice is to use modify in all cases to stay on the safe side.

Thanks for your quick answer Joaquin. I have added some more information to the existing content. — Ansh, Nov 21 '19 at 00:22
Thanks you for your quick answers and thank you for pointing out the error in the traversal. Is there anything we can find the issue from the back trace and confirm what is happening in the application on erase when the size is more and what boost library is try to do and leading to crash? Can this erase crash be avoided with exceptions? — Ansh, Nov 21 '19 at 14:37
Last frame of back trace pointing to /usr/include/boost/multi_index/detail/hash_index_node.hpp:167. That is when erasing the element with hashed index. Can this be observed when ordered_non_unique key on the map element is changing without using modify call, erase using hashed index that is find and erase give issues. Can we except what might be happening in the app using these boost library calls printed from the back trace? The problem may go way if we properly modify the key but is there any way to confirm that is the problem for this erase using hashed index? — Ansh, Nov 21 '19 at 16:22
If the program crashes when erasing an element, then the internal data structure has been somehow corrupted. Changing elements without using `modify` potentially corrupts the data structure. So... Your best bet is to fix what you already know is buggy and retry. — Joaquín M López Muñoz, Nov 21 '19 at 18:31
Are there any sample code to capture the exceptions for erase with iterators. Is there any way to know the map data is corrupted by traversing the whole map? — Ansh, Nov 21 '19 at 19:51
`erase` doesn't throw any exception: data corruption leads to uncontrollable behavior, which is not what exceptions are about. As for detecting where your program's bug is, [invariant-checking mode](https://www.boost.org/libs/multi_index/doc/tutorial/debug.html#invariant_check) can help. — Joaquín M López Muñoz, Nov 21 '19 at 20:57
Thank Joaquin for your comments. I am trying to traverse the container using https://github.com/ruediger/Boost-Pretty-Printer . It is only printing 200 elements but map node count is around 6000. Is there any issue with this pretty printer? Is there a way to search the map using one key value in GDB command prompt or traverse whole map using different indexes? Is the above pretty printer work if one of the hashed unique or hashed non unique as README file was saying it doesn't support those indexes even in the latest revision. — Ansh, Nov 24 '19 at 00:00
I'm no expert in any of the technologies you mention, sorry. — Joaquín M López Muñoz, Nov 24 '19 at 11:40
Thanks you Joaquín M López Muñoz. Is there anyway to traverse and print the previous and next elements of the key you are searching for each index type (hashed unique, hashed non-unique, ordered non unique)? — Ansh, Nov 26 '19 at 01:09

multi_index container ordered_non_unique is not rearranging the order after updating the key

1 Answers1