0

I have been trying a few solutions using the boost interprocess library with a map and now a multi_index_container in shared memory. With the multi_index_container, is there any way other than to iterate over the values returned from equal_range. I was hoping to retrieve a subset of results from my non_unique index (station name e.g. ST0012345) and then find/get the actual measurement type I need (e.g. Temperature).

I need to retrieve the unique pointID value for the measurement point (e.g. ST0012345SMD10000456.VoltsA = pointID - 45789) to insert the measurement value into a data Historian. I liked the idea of multi_index_container as our message payloads contain ~100 to 200 measurements for the one station in an array so I thought I could do one call to the shared memory container which will contain 500,000+ items and then lookup each measurement point from a much smaller list using the long unique string name.

It seems from the reading I have done that I may only be able to iterate over the smaller list returned from the multi_index_container rather than do a get/find.

If that is the case, am I better off to stick to my original shared memory map solution (which I have working) which as I said contains 500,000+ items of longish strings to match on to retrieve the pointID required by our data historian. There is a high data rate with 200 points being processed per second (I found my data map lookup could reach 2000 lookups per second in a test environment).

Also, if I do use the map, is there any harm in having several maps in shared memory e.g. a map for Sydney stations (~300,000 points), a map for Newcastle stations (~200,000 points).

Below is the code for my program which is retrieving items from a seperate process that created the multi_index_container.

using namespace boost::interprocess;
using namespace System;

namespace bmi = boost::multi_index;

typedef managed_shared_memory::allocator<char>::type              char_allocator;
typedef basic_string<char, std::char_traits<char>, char_allocator>shm_string;

struct tag
{
    shm_string name;
    shm_string stn;
    shm_string mtype;
    int         id;
    tag( const char *name_
      , const char_allocator &a
      , const char *stn_
      , const char_allocator &b
      , const char *mtype_
      , const char_allocator &d
      , int id_)
      : name(name_, a),
        stn(stn_, b),
        mtype(mtype_, d),
        id(id_)
    {}
};

struct name{};
struct stn{};
struct mtype{};
struct id{};

typedef bmi::multi_index_container<
  tag,
  bmi::indexed_by<
    bmi::ordered_unique
      <bmi::tag<name>,  BOOST_MULTI_INDEX_MEMBER(tag,shm_string,name)>,
    bmi::ordered_non_unique<
      bmi::tag<stn>,BOOST_MULTI_INDEX_MEMBER(tag,shm_string,stn)> >,
  managed_shared_memory::allocator<tag>::type
> tag_set;


typedef tag_set::iterator iterator;


int main(array<System::String ^> ^args)
{
   try{

      managed_shared_memory segment(open_only,"MySharedMemory");

      offset_ptr<tag_set> es = segment.find<tag_set>("My MultiIndex Container").first;

      char_allocator alloc_inst(segment.get_allocator<char>());
      shm_string  key_object(alloc_inst);
      key_object = "ST0012345SMD10000456";

      std::pair<tag_set::index<stn>::type::iterator, tag_set::index<stn>::type::iterator> values = es->get<stn>().equal_range(key_object);
      while(values.first != values.second) { 
          std::cout << values.first->name << " -- " <<"  ("<<values.first->stn<<","<<values.first->mtype<<")\n"; 
          ++values.first; 
      }


      char_allocator alloc_inst2(segment.get_allocator<char>());
      shm_string  key_object2(alloc_inst2);
      key_object2 = "ST0012345SMD10000456.VoltsA";

      ///// is there any way something like the following can be done rather than the above iterator ?????
      iterator it = values->get<name>(key_object2);  <-------  ????????????????????????
   }
   catch(const std::exception &exc){
       std::cerr << "Exception caught: " << exc.what();
      throw;
   }
   Sleep(3000);
   return 0;

}

Langfo
  • 430
  • 5
  • 17
  • So, what's the question? – Igor R. Nov 22 '13 at 06:12
  • The values returned from the equal_range, is there a way to then do a find or get on those values using the unique key or can I only iterate over the values? – Langfo Nov 22 '13 at 09:50
  • You certainly can apply any [standard algorithm](http://www.cplusplus.com/reference/algorithm/) on the pair of iterators you get. – Igor R. Nov 22 '13 at 14:25
  • Thanks Igor, So the "find" from the standard algorithm's (according to the link you provided) is just iterating over the elements in the range so I may as well just stick to iterating over the std::pair returned from the boost equal_range. – Langfo Nov 25 '13 at 04:50

1 Answers1

0

Consider using a composite key on the station name and temperature (which index you can use for lookups based on the station name alone or in ops involving the two keys).

Joaquín M López Muñoz
  • 5,243
  • 1
  • 15
  • 20
  • Hi Joaquin, this look up needs to handle 200 lookups per second. For 500,000 records, would I be better off sticking to a simple shared memory map and find the exact measurement point each lookup or keep heading down the multi-index path which uses a composite key as you suggest or a non_unique key which retrieves a range values which I then iterate over to find the exact measurement point. Also as another solution, is it possible to construct many maps in shared memory e.g. segment.construct("map1"), segment.construct("map2"), .... (mapN) ? – Langfo Nov 25 '13 at 05:01
  • Well, I'd say you'll have to profile to know which is your best option. Querying a 500 kelement container in less than 5ms doesn't look to me incredibly tight, though (I've made a simple test with a set of 700 kints on an Intel Core i5-2520M @2.50GHz and queries took around 7 us). – Joaquín M López Muñoz Nov 25 '13 at 15:18