0

Experiment

hping -S --data 100 -c 100 10.56.1.2 -i u100 round-trip min/avg/max = 0.5/14.0/15.8 ms

hping -S --data 100 -c 100 10.56.1.2 -i u1000 round-trip min/avg/max = 0.3/0.4/4.8 ms

Observation: When the packet send rate increases from 1 millisecond to 100 microseconds the avg latency increases 35 times (from .4 to 14) milliseconds

Experiment setup

Hardware: The source and destination servers are Supermicro (http://www.supermicro.com/products/system/1U/6016/SYS-6016TT-IBXF.cfm)

OS: RHEL6

Network: Connected using HP Procurve (model 2524 aka J4813A)

IP Addresses: Source server -> 10.56.1.1 and destination server -> 10.56.1.2

Kernel version: 2.6.32-358.2.1.el6.x86_64 on both source and destination.

Network card: Intel® 82576 Dual-Port Gigabit Ethernet Controller

Sysctl settings: no changes from the default

Can you solve this mystery ?

Vikas kedia
  • 11
  • 1
  • 3
  • Do you see this when you bypass the switch, i.e. connect the two servers directly together? – Bryan May 15 '13 at 07:03
  • These two servers are in a remote stock exchange data center, it is hard for me to run the experiment bypassing the switch. – Vikas kedia May 15 '13 at 07:10
  • What kernel version are you running? Any special sysctl settings? What NIC devices are you using? – ewwhite May 15 '13 at 15:28
  • I edited the original experiment setup section and added the details for 1. Kernel version 2. network card 3. sysctl settings. – Vikas kedia May 15 '13 at 20:16
  • Queuing latency? any packets lost? what about other data arriving? – hookenz May 15 '13 at 22:30
  • Q1) Queuing latency? I do not know how to measure this. Q2) Any packets lost? No Q3) What about other data arriving? None. – Vikas kedia May 20 '13 at 04:46

1 Answers1

0

This is RHEL6, so you have a lot of options...

Tuning Linux systems for financial trading is a broad topic. I can give you a few tips, but much of this will depend on your environment, application and your equipment.

You haven't changed any sysctl.conf settings, so there's a lot to do from that perspective. Have you monitored server CPU utilization and load while running your test?

Why don't you try a quick change to the two servers to help set a baseline...

Try setting the tuned-adm profile for the systems to: latency-performance

Also disable C-States and power-saving in the BIOS...

Another good guide to read to get started.

ewwhite
  • 197,159
  • 92
  • 443
  • 809