2

I have a bimap of length approx. 280000, and I am searching this bimap for atleast 18 million times for a value. The minimal example of the bimap that I have with me is given below;

#include <string>
#include <iostream>
#include <utility>
#include <boost/bimap.hpp>
#include <boost/bimap/unordered_set_of.hpp>
#include <boost/bimap/unordered_multiset_of.hpp>

namespace bimaps = boost::bimaps;
typedef boost::bimap<bimaps::unordered_set_of<unsigned long int>,
        bimaps::unordered_multiset_of<unsigned long int > > bimap_reference;
typedef bimap_reference::value_type position;
bimap_reference numbers;

int main()
{
    numbers.insert(position(123456, 100000)); // inserting in the bimap
    numbers.insert(position(234567, 80000));
    numbers.insert(position(345678, 100000));
    numbers.insert(position(456789, 80000));

    using ritr = bimap_reference::right_const_iterator;
    std::pair<ritr, ritr> range = numbers.right.equal_range(80000);
    auto itr = range.first;
    std::cout<<"first: "<<itr->first<<std::endl;
    if(itr != numbers.right.end() && itr->second ==80000){
        for (itr = range.first; itr != range.second; ++itr)
        {
            std::cout<<"numbers:"<<itr->second<<"<->"<<itr->first<<std::endl;
        }
    }
    else {
        std::cout<<"Not found:"<<std::endl;
    }
    return 0;
}

The searching of bimap 18 million times takes about 7 seconds. I would like to know how to improve the searching time. The other is, I have unordered_set_of<> and unordered_multiset_of<>, which helps me to create the bimap faster than If I use set_of<> and multiset_of<>, the search time is approx. same with both the cases. I also want to extend the length of the bimap to 170 million and searching will be be approx. 500 million times. So, how can I improve the search time?.

unordered_map<> is not the solution because I want bidirectional access.

AwaitedOne
  • 992
  • 3
  • 19
  • 42
  • Searching in a map (internally using a binary tree) has complexity O(ld(n)). Searching in a hash table can reach O(1) with the appropriate hash function. – Scheff's Cat Feb 27 '17 at 06:16
  • can the hash function be improved? what kind of values do you store there? – Ap31 Feb 27 '17 at 06:24
  • I just found out that `unordered_set` seems to be something like a hash table. I guess even faster can be only a `vector` (assuming you have enough memory available). What about spllitting tables and do multi-threading? – Scheff's Cat Feb 27 '17 at 06:30
  • I myself was always very skeptical about hash functions until I read this article [Hash](http://www.burtleburtle.net/bob/hash/doobs.html) and adopted the [SpookyHash: a 128-bit noncryptographic hash](http://www.burtleburtle.net/bob/hash/spooky.html) algorithm which improved my sorting of 3D vertices for a mesh algorithm. – Scheff's Cat Feb 27 '17 at 06:39
  • @Scheff I doubt about `vector` to be fast – AwaitedOne Feb 27 '17 at 06:55
  • I was not really serious. `vector` would be fast if the key is the index. (Therefore my considerations about available memory.) Sorry, forget this. – Scheff's Cat Feb 27 '17 at 06:57
  • For this use case, I think `unordered_set<>` and `-multiset<>` already are the best choices. How do measure the time? If you did it with the `std::cout` calls, then I guess most of the time is spent there. – Rene Feb 27 '17 at 07:09
  • @Rene not by `std::cout`, but using `high_resolution_clock` – AwaitedOne Feb 27 '17 at 07:12
  • How is the bimap constructed, either loaded at once or does it growing over time? In the first case, multi-threading could be a possible solution for speed-up. You would not have to expect much communication overhead (slowing down due to mutex locks). – Scheff's Cat Feb 27 '17 at 07:40
  • Concerning the multi-threading idea: I do not know how to limit the search in a `bimap` to a (sub-)range. Thus, I would simply split the one map into multiple bimaps. [std::thread::hardware_concurrency()](http://www.cplusplus.com/reference/thread/thread/hardware_concurrency/) can provide the appropriate number. Then, each bimap can be searched by its own thread. Assuming n CPU cores, the speed-up should nearly reach n. And, thus no mutex is necessary, only a `join` for each search thread. – Scheff's Cat Feb 27 '17 at 07:56
  • 1
    Consider a sorted vector or something like [boost::flat_map/flat_multimap](http://www.boost.org/doc/libs/1_63_0/doc/html/container/non_standard_containers.html#container.non_standard_containers.flat_xxx). These could be faster because memory is allocated as contiguous block which improves cache hits. – zett42 Feb 27 '17 at 08:28
  • @Scheff bimap is constructed at first, once done, searching starts. – AwaitedOne Feb 27 '17 at 09:35
  • Strange that the code execution takes 7 seconds. The modified original code with 18 millions search operations takes on [coliru](http://coliru.stacked-crooked.com/a/f614af5475a844da) 0.4 s. – megabyte1024 Feb 27 '17 at 15:28
  • @megabyte1024 I wonder if you get the same performance on `bimap` with `280000` entries not `28000` . – AwaitedOne Feb 27 '17 at 16:06
  • No wonder, [the execution time is the same due to O(1), mentioned above](http://coliru.stacked-crooked.com/a/1d9f1172d53659ea). – megabyte1024 Feb 27 '17 at 19:06
  • @megabyte1024 shows `7.803099s wall, 7.750000s user + 0.060000s system = 7.810000s CPU (100.1%)`, `8.486985s wall, 8.470000s user + 0.020000s system = 8.490000s CPU (100.0%)`, and `7.803785s wall, 7.780000s user + 0.020000s system = 7.800000s CPU (100.0%)` on three runs. May be your system configuration is different? – AwaitedOne Feb 28 '17 at 04:50

0 Answers0