What is the fastest way to convert unsigned char array to IP string

Question

I need to process a lot of these in (more or less) real-time. The method I'm currently using is not cutting it anymore.

std::string parse_ipv4_address( const std::vector<unsigned char> & data, int start )
{
    char ip_addr[16];
    snprintf( ip_addr, sizeof(ip_addr), "%d.%d.%d.%d", 
        data[start + 0], data[start + 1], data[start + 2], data[start + 3] );
    return std::string( ip_addr ); 
}

// used like this
std::vector<unsigned char> ip = { 0xc0, 0xa8, 0x20, 0x0c };
std::string ip_address = parse_ipv4_address( ip, 0 );

std::cout << ip_address << std::endl; // not actually printed in real code 
// produces 192.168.32.12

Is there a faster way to do it? And how?

NOTE! Performance is the issue here so this issue is not a duplicate.

Answer 1

Here are the potential candidates that impact performance:

• snprintf needs to parse the format string, and performs error handling. Either costs time, to implement features you don't need.
• Constructing a std::string object on return is costly. It stores the controlled sequence in freestore memory (usually implemented as heap memory), that is somewhat costly to allocate in C++ (and C).
• Use of a std::vector to store a 4-byte value needlessly burns resources. It, too, allocates memory in the freestore. Replace that with char[4], or a 32-bit integer (uint32_t).

Since you don't need the versatility of the printf-family of functions, you might drop that altogether, and use a lookup-table instead. The lookup table consists of 256 entries, each of which holding the visual representation of the byte values 0 through 255. To optimize this, have the LUT contain a trailing . character as well. (Special care needs to be taken, to address endianness. I'm assuming little-endian here.)

A solution might look like this¹⁾:

const uint32_t mapping[] = { 0x2E303030, // "000."
    0x2E313030, // "001."
    // ...
    0x2E343532, // "254."
    0x2E353532  // "255."
};

alignas(uint32_t) char ip_addr[16];
uint32_t* p = reinterpret_cast<uint32_t*>(&ip_addr[0]);
p[0] = mapping[data[0]];
p[1] = mapping[data[1]];
p[2] = mapping[data[2]];
p[3] = mapping[data[3]];

// Zero-terminate string (overwriting the superfluous trailing .-character)
ip_addr[15] = '\0';

// As a final step, waste all the hard earned savings by constructing a std::string.
// (as an ironic twist, let's pick the c'tor with the best performance)
return std::string(&ip_addr[0], &ip_addr[15]);

// A more serious approach would either return the array (ip_addr), or have the caller
// pass in a pre-allocated array for output.
return ip_addr;

---

¹⁾ Disclaimer: Casting from char* to uint32_t* technically exhibits undefined behavior. Don't use, unless your platform (compiler and OS) provide additional guarantees to make this well defined.

Answer 2

Three Four answers for the price of one.

First, make really, really sure that you're optimizing the right part. Both std::vector and std::string creation involve memory allocations, and cout << could involve file access, graphics, etc!

Second: Don't use vector to represent the 4-bytes of an IP address. Just use char ip[4], or even a 32-bit integer

Third: I'm guessing that you're not dealing with totally random IP addresses. Probably a few hundred or thousand different addresses? In which case, use a std::map<INT32, std::string> as a cache, and just pull the required ones from the cache as required, writing new ones in as needed. If the cache gets too big, just empty it and start over...

---

Fourth: Consider writing the IP address in Hexadecimal dotted quad notation. This is still accepted by calls like inet_addr() and has several advantages: All fields are fixed width, there are only 8 'characters' to update, and the binary to Hex conversion is usually faster than binary to decimal. https://en.wikipedia.org/wiki/IPv4#Address_representations

Answer 3

Lookup table could be of use here (initialized on program start). I guess you already have profiling configured so I didn't profile solution and wonder what would be the results so please share when you get some.

char LOOKUP_TABLE[256][4];

void init_lookup_table() {
    char digits[] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9'};

    for (int i = 0; i < 10; ++i) {
        LOOKUP_TABLE[i][0] = digits[i % 10];
        LOOKUP_TABLE[i][1] = '\0';
        LOOKUP_TABLE[i][2] = '\0';
        LOOKUP_TABLE[i][3] = '\0';
    }

    for (int i = 10; i < 100; ++i) {
        LOOKUP_TABLE[i][0] = digits[(i / 10) % 10];
        LOOKUP_TABLE[i][1] = digits[i % 10];
        LOOKUP_TABLE[i][2] = '\0';
        LOOKUP_TABLE[i][3] = '\0';
    }
    for (int i = 100; i < 256; ++i) {
        LOOKUP_TABLE[i][0] = digits[(i / 100) % 10];
        LOOKUP_TABLE[i][1] = digits[(i / 10) % 10];
        LOOKUP_TABLE[i][2] = digits[i % 10];
        LOOKUP_TABLE[i][3] = '\0';
    }
}

void append_octet(char **buf, unsigned char value, char terminator) {
    char *src = LOOKUP_TABLE[value];
    if (value < 10) {
        (*buf)[0] = src[0];
        (*buf)[1] = terminator;
        (*buf) += 2;
    }
    else if (value < 100) {
        (*buf)[0] = src[0];
        (*buf)[1] = src[1];
        (*buf)[2] = terminator;
        (*buf) += 3;
    }
    else {
        (*buf)[0] = src[0];
        (*buf)[1] = src[1];
        (*buf)[2] = src[2];
        (*buf)[3] = terminator;
        (*buf) += 4;
    }
}

std::string parse_ipv4_address( const std::vector<unsigned char> & data, int start ) {
    char ip_addr[16];
    char *dst = ip_addr;
    append_octet(&dst, data[start + 0], '.');
    append_octet(&dst, data[start + 1], '.');
    append_octet(&dst, data[start + 2], '.');
    append_octet(&dst, data[start + 3], '\0');
    return std::string( ip_addr ); 
}


int main() {
    init_lookup_table();

    std::vector<unsigned char> ip = { 0xc0, 0x8, 0x20, 0x0c };
    std::cout << parse_ipv4_address( ip, 0 ) << std::endl;
}

Other way to improve performance would be to replace string with specialized object. In that case you will be able to implement required I/O methods (my guess is that you need string to print it somewhere) and will be freed from copying on string construction.

UPD on second thought I guess in my code lookup table is out of use so one could just copy code used to build lookup table to append_octet directly making digits global.

Updated code (thanks to MikeMB and Matteo Italia) which also looks very cache friendly

inline void append_octet(char **buf, unsigned char value, char terminator) {
    if (value < 10) {
        (*buf)[0] = '0' + (value % 10);
        (*buf)[1] = terminator;
        (*buf) += 2;
    }
    else if (value < 100) {
        (*buf)[0] = '0' + ((value / 10) % 10);
        (*buf)[1] = '0' + (value % 10);
        (*buf)[2] = terminator;
        (*buf) += 3;
    }
    else {
        (*buf)[0] = '0' + ((value / 100) % 10);
        (*buf)[1] = '0' + ((value / 10) % 10);
        (*buf)[2] = '0' + (value % 10);
        (*buf)[3] = terminator;
        (*buf) += 4;
    }
}

std::string parse_ipv4_address( const std::vector<unsigned char> & data, int start ) {
    char ip_addr[16];
    char *dst = ip_addr;
    append_octet(&dst, data[start + 0], '.');
    append_octet(&dst, data[start + 1], '.');
    append_octet(&dst, data[start + 2], '.');
    append_octet(&dst, data[start + 3], '\0');
    return std::string( ip_addr ); 
}


int main() {
    std::vector<unsigned char> ip = { 0xc0, 0x8, 0x20, 0x0c };
    std::cout << parse_ipv4_address( ip, 0 ) << std::endl;
}

UPD 2 I guess I found a way to avoid extra copy (altough there's still extra copy on return). Here's versions with look up table and w/o it

#include <string>
#include <iostream>
#include <vector>

std::string LUT[256];

void init_lookup_table() {
    for (int i = 0; i < 10; ++i) {
        LUT[i].reserve(2);
        LUT[i].push_back('0' + i);
        LUT[i].push_back('.');
    }
    for (int i = 10; i < 100; ++i) {
        LUT[i].reserve(3);
        LUT[i].push_back('0' + (i/10));
        LUT[i].push_back('0' + (i%10));
        LUT[i].push_back('.');
    }
    for (int i = 100; i < 256; ++i) {
        LUT[i].reserve(4);
        LUT[i].push_back('0' + (i/100));
        LUT[i].push_back('0' + ((i/10)%10));
        LUT[i].push_back('0' + (i%10));
        LUT[i].push_back('.');
    }
}

std::string parse_ipv4_address_lut( const std::vector<unsigned char> & data, int start ) {
    std::string res;
    res.reserve(16);
    res.append(LUT[data[start + 0]]);
    res.append(LUT[data[start + 1]]);
    res.append(LUT[data[start + 2]]);
    res.append(LUT[data[start + 3]]);
    res.pop_back();
    return res; 
}

inline void append_octet_calc(std::string *str, unsigned char value, char terminator) {
    if (value < 10) {
        str->push_back('0' + (value % 10));
        str->push_back(terminator);
    }
    else if (value < 100) {
        str->push_back('0' + ((value / 10) % 10));
        str->push_back('0' + (value % 10));
        str->push_back(terminator);
    }
    else {
        str->push_back('0' + ((value / 100) % 10));
        str->push_back('0' + ((value / 10) % 10));
        str->push_back('0' + (value % 10));
        str->push_back(terminator);
    }
}

std::string parse_ipv4_address_calc( const std::vector<unsigned char> & data, int start ) {
    std::string res;
    res.reserve(16);
    append_octet_calc(&res, data[start + 0], '.');
    append_octet_calc(&res, data[start + 1], '.');
    append_octet_calc(&res, data[start + 2], '.');
    append_octet_calc(&res, data[start + 3], '\0');
    return res; 
}


int main() {
    init_lookup_table();

    std::vector<unsigned char> ip = { 0xc0, 0x8, 0x20, 0x0c };
    std::cout << parse_ipv4_address_calc( ip, 0 ) << std::endl;
    std::cout << parse_ipv4_address_lut( ip, 0 ) << std::endl;
}

UPD 3 I made some measurements (1 000 000 repeats)

clang++ -O3
orig...done in 5053 ms // original implementation by OP
c_lut...done in 2083 ms // lookup table -> char[] -> std::string
c_calc...done in 2245 ms // calculate -> char[] -> std::string
cpp_lut...done in 2597 ms // lookup table + std::string::reserve + append
cpp_calc...done in 2632 ms // calculate -> std::string::reserve + push_back
hardcore...done in 1512 ms // reinterpret_cast solution by @IInspectable

g++ -O3
orig...done in 5598 ms // original implementation by OP
c_lut...done in 2285 ms // lookup table -> char[] -> std::string
c_calc...done in 2307 ms // calculate -> char[] -> std::string
cpp_lut...done in 2622 ms // lookup table + std::string::reserve + append
cpp_calc...done in 2601 ms // calculate -> std::string::reserve + push_back
hardcore...done in 1576 ms // reinterpret_cast solution by @IInspectable

Note that 'hardcore' solution doesn't equivalent because of leading zeroes.

Answer 4

you can use a lookup table which holds a string for numbers from 0 to 255. if speed is very important, you can also use inline keyword or maybe a macro for the function. also you can check sse instructions.

By the way, usually the more primitive your code the faster it is. I would use unsigned char array instead of vector, char array instead of string, memcpy(or even byte by byte copy directly) instead of sprintf.

Answer 5

Here you go...

    std::string IP_parse(unsigned char data[4])
    {
            std::string parsedString = "";
            snprintf((char*)parsedString.c_str(), sizeof(char[15]), "%d.%d.%d.%d", data[0], data[1], data[2], data[3]);
            return parsedString;
    }