0

I have a simple structure:

struct data
{
  bool flag;
  std::size_t ix;
  std::size_t iy;
  std::size_t d;
};

I keep all instances of this structure in container, and I need to search it as fast as possible. At the moment I use the std::set with custom comparison functor, based on lexicographical comparison. However, I want to experiment with storing my data in std::unordered_set.

I used boost::hash_combine for creating custom hash functor, but the result was actually even slower, than with std::set. I want now to pack all the data from the structure in a unsigned long long and to use standard std::hash functor. So an idea is to have something like:

struct hashed_data
{
  union
  {
    struct
    {
      bool flag;
      std::size_t ix;
      std::size_t iy;
      std::size_t d;
    } data;
    unsigned long long value_to_hash;
  };
};//hashed_data

The problem is, I want to make sure, that my structure actually fits in unsigned long long type. I can change the type of my data members within limits:

ix and iy are less then a 100,000 (they might not fit into 16 bits integer, but 24 bits is more than enough, however, there is no standard 24 bits integers?).

d can safely be of a char type.

I would be grateful for any views on this problem -- whether idea is sound at all and how to achieve my objective.

timrau
  • 22,578
  • 4
  • 51
  • 64
one_two_three
  • 501
  • 2
  • 10
  • 1
    You can use bit fields to pack your fields into 8 bytes. But the `union`-based type punning will cause UB either way, instead you need keep it a `struct`, and `memcpy` into `unsigned long long` when needed. – HolyBlackCat Jul 17 '20 at 14:45
  • The idea that an `unordered_set` will always be faster than a `set` is a myth. It depends on many things. You can't focus _just_ on algorithmic complexity. It sounds to me like you've already discovered, by actual profiling, that your case is best served by a `set`. So why not just stick with it? – Asteroids With Wings Jul 17 '20 at 15:24
  • Although, as a further suggestion, have you tried maintaining a sorted vector? That is, start with an empty vector (or a large pre-existing dataset that you then `std::sort`), then do `find`s to locate insertion and erasure positions as needed. Yes, you will be shuffling data each time, but copying blocks of memory can be fast and with primitives like yours you may find the gain of cache locality (and the loss of a ton of dynamic allocations) far outweighs the disadvantages. – Asteroids With Wings Jul 17 '20 at 15:27
  • @Asteroids With Wings -- this is a time-critical applications, so I try different angles. Also, if I pack all into a standard-type single number, I might use this number for comparison to fill the ```std::set``` instead of using lexicographical comparison on separate structure members. – one_two_three Jul 17 '20 at 15:29
  • Worth a try, but I doubt the comparisons are your bottleneck. Every element in your set has to be dynamically allocated. It's expensive. – Asteroids With Wings Jul 17 '20 at 15:31
  • @Asteroids With Wings -- the most frequently used operation - to find whether given instance of the structure is already present in the container. I do not think sorted vector will improve the speed here. – one_two_three Jul 17 '20 at 15:37
  • @one_two_three I guarantee that you can lookup faster in a sorted vector than in a set ;) (though the difference will be negligible unless you do this a lot). The operation is completely identical semantically, and in terms of implementation is different only in the data being found in one contiguous block (great for caching!). The disadvantages come when you need to erase or insert (which requires shuffling elements around). But I still think you could find those to be acceptable overall, when balanced against the benefits. You'd need to measure it. – Asteroids With Wings Jul 17 '20 at 15:41
  • @Asteroids With Wings thank you for suggestion, I shall try it. Even so -- the appeal of packing everything in a single number, which can be compared with a single operation, rather then four, holds even for this case. – one_two_three Jul 17 '20 at 17:42
  • @one_two_three If that weighs up well against the cost of packing and unpacking, sure. Again, you need to _measure_ rather than guess. Integer comparisons are very fast and your compiler knows how to do things like this optimally. – Asteroids With Wings Jul 19 '20 at 19:21

0 Answers0