-1

I have one message which contains a unique id, and some information I need to send through MPI processes. To do so, I convert this message to an array of bits.

I convert one message to bit representation by using the std::bitset class. Now, I want to send it to another process with MPI.

I can use the function std::bitset::to_string() to convert each bit to a char; but the size of the message will increase to sizeof(char)*MSG_SIZE (where MSG_SIZE is equal to 256 in my case).

static const int MSG_SIZE = 256;
std::bitset<MSG_SIZE> msg;
msg = convert_to_bit(uint64_t uid, {additional information...});
// Using the function to_string(), my message is now of size (MSG_SIZE * sizeof(char))
// because each bit in the bitset is represented by a char (= 32 bits)
MPI_Send(msg.to_string().c_str(), msg.to_string().size(), MPI_BYTE, 1, 0, MPI_COMM_WORLD);

How can I avoid this situation, keep the size of the message equals to 256 bits ?

In fact, I would like a situation like this:

static const int MSG_SIZE = 256;
std::bitset<MSG_SIZE> msg;
msg = convert_to_bit(uint64_t uid, {additional information...});
// not necessary represented using char
// However I have no idea about which type I have to use
char* msg_in_bits = new char[MSG_SIZE / sizeof(char)];
msg_in_bits = do_something(msg);
MPI_Send(msg_in_bits, MSG_SIZE, MPI_BYTE, 1, 0, MPI_COMM_WORLD);

I only want to send a message of the real size of my message: MSG_SIZE = 256 bits. Not to increase the size of my message because I will represent each bit by a char (= 32bits). I want to represent a bit... by a bit, not a char.

Thank you

Adrien Roussel
  • 39
  • 1
  • 6
  • 5
    It is unclear what you are asking. Please [edit] your post to clarify. – Ron Aug 21 '18 at 09:03
  • 1
    you have a message of 256 bits size, when you do some converions then the size increases, but you dont want the size to increase... what about not doing the conversion? Maybe some code would help to understand what you are doing... – 463035818_is_not_an_ai Aug 21 '18 at 09:05
  • Please [edit] your question to include a [mcve] – Tas Aug 21 '18 at 09:18
  • Thanks for the reply, I edited my question while hoping that it is now clear for you ! – Adrien Roussel Aug 21 '18 at 09:20
  • 1
    @AdrienRoussel There seems to be a lot of confusion in your sample code about sizes. `sizeof(char)` is 1 not 8, the second parameter to MPI_Send is a size in bytes not in bits. – john Aug 21 '18 at 09:27
  • 2
    _I convert one message to bit representation by using the std::bitset class._ Why? How? A character consists of 8 bits (usually). If only a sub-set of characters is used less bits might be sufficient as well. (You don't tell.) My first idea was, that it might be required for `MPI_Send()` but you send `MSG_SIZE*sizeof(char), MPI_BYTE` i.e. 256 bytes. This is somehow confusing... – Scheff's Cat Aug 21 '18 at 09:28
  • 1
    You want to send an array of bytes, each byte packing 8 (or CHAR_BIT) bits of infirmation. Unfortunately `std::bitset` doesn't support conversion to such representation. My advice is to avoid `std::bitset` for this application. Use `std::array` (calculate the needed size) and provide conversion to and from message yourself. – n. m. could be an AI Aug 21 '18 at 09:31
  • Why do you need to convert to bits? – Killzone Kid Aug 21 '18 at 09:44
  • @phuclv this works only if ulong has enough bits to represent the value. Not many systems have a 256-bit long ulong. – n. m. could be an AI Aug 21 '18 at 09:46
  • @n.m. I know. I mean you can loop through the bitset with `(x & ~0UL).to_ulong(); x >>= 64` to extract all the bits – phuclv Aug 21 '18 at 09:48
  • @phuclv yes you can. You would also need to mask out the upper bits to prevent overflow. That's a lot of bit shuffling and copying stuff around though. You can achieve the same functionality if you use an array of bytes (or ulongs) in the first place, with about same amount of source code, but no copying and much less bit shuffling. – n. m. could be an AI Aug 21 '18 at 09:55
  • @n.m. I would like to do so, however I am not convenient with this kind of representation. For example, I have one uint64_t to represent the unique identifier of my message but I do not know how to access to its bit representation and then store it in an std::array for example. That is why I used the bitset class, because it was easier for me – Adrien Roussel Aug 21 '18 at 09:56
  • @phuclv each bit in the bitset *converted to a string* is represented by a char. I'm not sure where 32 comes from though. – n. m. could be an AI Aug 21 '18 at 09:56
  • @phuclv not when you convert it as an array of char with the to_string() function, each bit is then converted as a char – Adrien Roussel Aug 21 '18 at 09:57
  • @AdrienRoussel no, that's the string output when you convert the bitset to string. I didn't say that's the way to transfer bits. In fact it's the worst method ever. "I don't know where you got `because each bit in the bitset is represented by a char (= 32 bits)" is incorrect. Because each bit in the bitset is obviously represented internally by... a bit. And I don't know how bitset is easier for you when you have to assign 64 bits? Using a char array you just need to break the long into bytes and combine them back at the other end – phuclv Aug 21 '18 at 09:58
  • @phuclv converting a uint64_t with bitset looks like this: uint64_t x = 64; std::bitset<64> x_bit(x); However, I do not know how bitset convert x into bits because I miss some knowledge about bit manipulation – Adrien Roussel Aug 21 '18 at 10:03
  • 1
    @AdrienRoussel and the simplest way to copy it to an array is `memcpy(&msg, &x, sizeof x)`. Or even better, use `struct {uint64_t id; uint8_t type; uint8_t [SIZE] data } msg; msg.id = x;`. If you don't know about bitwise operations then you should learn about it first. Only use bitset if you understand how it works – phuclv Aug 21 '18 at 10:05
  • to convert from bitset to int use [`to_ulong()` and `to_ullong()`](https://en.cppreference.com/w/cpp/utility/bitset/to_ulong) as I commented above. But as I said, it's not the correct way to solve the real problem. This is an [XY problem](https://meta.stackexchange.com/q/66377/230282) – phuclv Aug 21 '18 at 10:15

2 Answers2

1

Something like this, not the only way to do it

#include <cstdint>

static const int MSG_SIZE = 256;
static const int MSG_SIZE_IN_BYTES = MSG_SIZE/8;
std::bitset<MSG_SIZE> msg = ...;
uint8_t msg_in_bits[MSG_SIZE_IN_BYTES] = {0};
for (int i = 0; i < MSG_SIZE; ++i)
    if (msg[i])
        msg_in_bits[i/8] |= 1 << (i%8);
MPI_Send(msg_in_bits, MSG_SIZE_IN_BYTES, MPI_BYTE, 1, 0, MPI_COMM_WORLD);
john
  • 85,011
  • 4
  • 57
  • 81
  • Do you think that it should be smarter to directly pass by an array of uint8_t instead of passing by the bitset class and then convert it to this representation ? – Adrien Roussel Aug 21 '18 at 09:29
  • 1
    Whichever representation is most convenient. Without knowing your app I can't really say. – john Aug 21 '18 at 09:31
  • In fact, my message contains a message type (4 bits), unique identifier (64 bits), a granularity size (16 bits), and a function name (172 bits). And I want to convert all these informations in bits to then send it via MPI. First, I used std::bitset because it was easy for me to use this. But maybe it should be more reasonable to use uint8_t instead – Adrien Roussel Aug 21 '18 at 09:39
  • 1
    `MSG_SIZE/8` ==> `(MSG_SIZE+7)/8`, just to be on the safe side. – n. m. could be an AI Aug 21 '18 at 09:48
  • 1
    how to you store the function name? 172 is not even divisible by 8. Using bitset only makes sense if you're working with the bits. Here all your data fields are a lot bigger than a single bits, thus dealing with individual bits will be slower and more error prone – phuclv Aug 21 '18 at 10:03
0

If you just want to send a std::string msg with mpi, I would do something like this

MPI_Send(msg.c_str(), msg.size(), MPI_CHAR, 0, 0, MPI_COMM_WORLD);

I think this is less error prone and not slower than your approach.

Is there any reason you like to convert it first?

schorsch312
  • 5,553
  • 5
  • 28
  • 57
  • Because of memory space, and communication speed. I do not want to spend much more time to send more bits than I use – Adrien Roussel Aug 21 '18 at 09:36
  • I am sorry. I don't understand why you send more than you use. – schorsch312 Aug 21 '18 at 09:44
  • When I convert my array of bits from bitset with the function to_string(), each bit is represented as a char (sizeof(char) = 32 bits). So the size of the message is then (MSG_SIZE * 32 bits) instead of MSG_SIZE bits – Adrien Roussel Aug 21 '18 at 09:51
  • @AdrienRoussel `sizeof(char)` is 1, which contains `CHAR_BIT` bits (most commonly 8) – phuclv Aug 21 '18 at 09:52