10

I have a very simple wrapper for boost::asio sockets sending multicast messages:

// header
class MulticastSender
{
public:

    /// Constructor
    /// @param ip - The multicast address to broadcast on
    /// @param port - The multicast port to broadcast on
    MulticastSender(const String& ip, const UInt16 port);

    /// Sends a multicast message
    /// @param msg - The message to send
    /// @param size - The size of the message (in bytes)
    /// @return number of bytes sent
    size_t send(const void* msg, const size_t size);

private:

    boost::asio::io_service m_service;
    boost::asio::ip::udp::endpoint m_endpoint;
    boost::asio::ip::udp::socket m_socket;

};

// implementation
inline MulticastSender::MulticastSender(const String& ip, const UInt16 port) :
    m_endpoint(boost::asio::ip::address_v4::from_string(ip), port),
    m_socket(m_service, m_endpoint.protocol())
{
    m_socket.set_option(boost::asio::socket_base::send_buffer_size(8 * 1024 * 1024));
    m_socket.set_option(boost::asio::socket_base::broadcast(true));
    m_socket.set_option(boost::asio::socket_base::reuse_address(true));
}

inline size_t MulticastSender::send(const void* msg, const size_t size)
{
    try
    {
        return m_socket.send_to(boost::asio::buffer(msg, size), m_endpoint);
    }
    catch (const std::exception& e)
    {
        setError(e.what());
    }
    return 0;
}

// read and send a message
MulticastSender sender(ip, port);
while(readFile(&msg)) sender.send(&msg, sizeof(msg));

When compiled on Windows 7 using Visual Studio 2013, I get throughput of ~11 MB/s, on Ubuntu 14.04 ~100 MB/s. I added timers and was able to validate the send(...) method is the culprit.

I tried with and without antivirus enabled, and tried disabling a few other services with no luck. Some I cannot disable due to permissions on the computer, like the firewall.

I assume there is a service on Windows running that is interfering, or my implementation is missing something that is effecting the application on Windows and not Linux.

Any ideas on what might be cauing this would be appreciated

steveo225
  • 11,394
  • 16
  • 62
  • 114
  • I'm assuming you compiled with proper optimizations? – Johan Lundberg Nov 16 '15 at 18:56
  • Yes, no debug information, /O2 /Ot – steveo225 Nov 16 '15 at 19:00
  • How about the NIC settings on windows ? I am not sure but it may not have optimized setting( s) for this usecase – Arunmu Nov 16 '15 at 19:30
  • They would be the defaults, not sure what they should be set to for optimal multicast sends – steveo225 Nov 16 '15 at 19:34
  • 1
    Have you tried doing this without boost? Making direct calls through the Win32 API? I would assume you would see the same, but perhaps its a faulty implementation in boost? – Matthew Hoggan Nov 18 '15 at 20:01
  • Tuning sockets is a hard job. It could be anything. I would suggest to get some common network performance tool and repeat you measurements. Most likely problem is not in aiso. – VladimirS Nov 18 '15 at 20:17
  • @MatthewHoggan No, I have not. I need to have a version that works both in Linux and Windows, that is why I chose boost, to only have to manage 1 version – steveo225 Nov 18 '15 at 20:48
  • I have no idea how to profile on windows. Is there a way to profile userspace up to the relevant system call? – Jason Nov 18 '15 at 21:02
  • @steveo225 Understood, but trying to pinpoint the issue might help better resolve where the issue lies. – Matthew Hoggan Nov 18 '15 at 22:26
  • @Jason VS2015 has a build in profiler. I don't know if it shows issues in Windows specific libraries, but if the problem lies within your application or one of it's dlls the profiler is pretty good. To got to it go to Debug --> Show Diagnostic Tools (Ctrl+Alt+F2). – Matthew Hoggan Nov 18 '15 at 22:28
  • @MatthewHoggan Netcat (MSYS2?) might be an easy way to test boost as a cause. Normally in linux, I would start with `perf` and then move to `ftrace`. I don't know if there are windows equivalents, but it's probably easiest to diagnose from userspace down to kernel (possibly hardware and network). – Jason Nov 18 '15 at 22:55
  • I'd be interested to see what's inside this readFile function. Have you tested how many MB/s you can read from the file? Maybe try just a simpe for loop sending the same data over and over(without the readFile function), and see if that speeds up your throughput at all. Also, is the file static, or is there data being written to the file as it is being read? – wizurd Nov 21 '15 at 03:58
  • @wizurd As stated in the post, I validated `send` was the taking all the time. To be completely sure, I changed `readFile` to be a noop. It is not to blame – steveo225 Nov 24 '15 at 12:35

2 Answers2

2

Is windows and ubuntu running on the same machine?

If not, it seems that your windows machine is limited by 100Mbit Ethernet, while the ubuntu machine seems to work with 1Gbit Ethernet.

(In case thats not the cause of the problem, i am sorry for posting an anwser instead of commenting. But i am not able to do so and your code is that simple and the data rates are so obvious [11*8MB/s ~ 100Mbit/s and 100MB/s ~ 800Mbit/s]. I just had to make that hint...)

MarkusAtCvlabDotDe
  • 1,032
  • 5
  • 12
  • They are different machines, but both have gigabit network cards. But you do bring up an interesting point about the rates. I am curious now if there is something causing the Windows machine to only allow UDP/Multicast to run at 100 Mbit speeds – steveo225 Nov 24 '15 at 13:30
  • Are you using Gigabit Ethernet cables for both machines? And a very important aspect: Have you set the network throttling index to 0xffffffff!? That can absolutely cause UDP package drops on your windows machine. – MarkusAtCvlabDotDe Nov 24 '15 at 14:17
  • Yes, the cables are sufficient. I get rates in excess of 70MB/s to a NAS. I changed the network throttling index to 0xffffffff with no change. – steveo225 Nov 24 '15 at 15:59
0

If you data transfer if huge say more that 10 MB messages i would suggest you to use TCP instead of UPD/Multicast. TCP is a reliable protocol.

I read in a case where a stream of 300 byte packets was being sent over Ethernet (1500 byte MTU) and TCP was 50% faster than UDP. Because TCP will try and buffer the data and fill a full network segment thus making more efficient use of the available bandwidth but UDP puts the packet on the wire immediately thus congesting the network with lots of small packets. In windows i suggest you to use TCP over UDP/Multicast.

Murali Manohar
  • 589
  • 7
  • 35
  • Can't use TCP. Multiple receivers are listening for the data. I don't need to rework the source as it works perfectly on Linux, my only issue is that on Windows it is slower, too slow. – steveo225 Nov 25 '15 at 12:31
  • Also worth mentioning, the data being sent is buffered in a similar way to TCP such that the payload size is close to the MTU of the network. Currently, I think the packets being sent are 1372 bytes each – steveo225 Nov 25 '15 at 17:48