Estimating network I/O speed using scp

Question

I want to have a rough estimate of the network I/O speed between two linux servers. The problem is that I don't have sudo access to the servers. So I tried transferring a 1GB file between the two servers using scp. I suppose that with the encryption there will be some slowdown. How much slowdown should I be expecting? Also can the scp bandwidth usage be capped by the server admin? How do I know if it is capped?

Answer 1

The encryption is normally not the bottleneck in a scp transfer, but you can use ftp instead.

My usual way is to open a Python web server on any directory on a certain port using this single command

python -m SimpleHTTPServer 8000  # python 2
python -m http.server 8000 # python 3

And on the other side just use wget to download it

wget http://[ip address]:8000/[some big file]

Any network activity could be limited by the server admin and the usual indicator is that your speed is maintained at a nice stable level (e.g. 500KB/s)

Answer 2

iperf is there for network performance testing, available in all good repositories, and plenty of articles for usage tips.

http://iperf.sourceforge.net/

Random usage articles:

http://www.nanog.org/meetings/nanog43/presentations/Dugan_Iperf_N43.pdf

http://maddhat.com/testing-network-performance-using-iperf-3

Using file transfer programs like scp or ftp brings in disk IO as a bottleneck source.

Answer 3

How much slowdown should I be expecting? Encryption should not be much overhead. Also, the purpose of this speed test is to measure sever to server I/O performance, so wouldn't you want that to reflect the typical type of communication, which would be with encryption?

Also can the scp bandwidth usage be capped by the server admin? How do I know if it is capped? Yes, this is possible at various tiers in the network. For example, using tc (example here), and at the router level. If you have sudo privileges you can check if tc limits are in place. Otherwise, you could try using a different machine without limits and see if the speed is different. If it's done at the router level you have no easy way of knowing.

Here's a bash script that will 1) check if a server is reachable, 2) create a file of a given size, 3) use scp to measure upload and optionally download rates, 4) delete the file locally and remotely.

Use it like speed_test.sh --server=1.2.3.4 --test_size=128 --do_download=1 --server_dir=.

Credit note: It is my modification of the script originally posted here.

#!/bin/bash

test_file=".scp-test-file"

# usage function
function usage()
{
   cat << HEREDOC
   Usage: $progname --server=<ip address> [--test_size=<MB> --do_download=<0/1> --server_dir=<server side dir>]
   optional arguments:
     -h, --help           show this help message and exit
HEREDOC
}

# initialize variables and defaults
progname=$(basename $0)
server=
server_dir=./
do_download=1
test_size=10

###
# all args
L=(server \
do_download \
test_size \
server_dir)

arg_parse_str="help"
for arg in "${L[@]}"; do
    arg_parse_str=${arg_parse_str},${arg}:
done
#echo $arg_parse_str

OPTS=$(getopt -o "h" --long ${arg_parse_str} -n "$progname" -- "$@")
if [ $? != 0 ] ; then echo "Error in command line arguments." >&2 ; usage; exit 1 ; fi
eval set -- "$OPTS"

while true; do
  # uncomment the next line to see how shift is working
  # echo "\$1:\"$1\" \$2:\"$2\""
  case "$1" in
    -h | --help ) usage; exit; ;;
    -- ) shift; break ;;
  esac
  found=
  for arg in "${L[@]}"; do
    if [ "$1" == "--$arg" ]; then
        declare ${arg}="$2";
        shift 2;
        found=1
        break
    fi
  done
  if [ -z "$found" ]; then
    break
  fi
done

if [ -z "$server" ]
then
    usage;
    exit;
fi

#[[ -z $(nc -W 1 -w 10 $server 22 2>/dev/null) ]] && echo "$server unreachable" && exit 0
#if your nc complains about not having -i command line option then use the line above
[[ -z $(nc -i 1 -w 10 $server 22 2>/dev/null) ]] && echo "$server unreachable" && exit 0


# generate a file of all zeros
echo "Generating $test_size MB test file..."
dd if=/dev/zero of=$test_file bs=$(echo "$test_size*1024*1024" | bc) \
  count=1 &> /dev/null
# upload test

echo "Testing upload to $server..."
up_speed=$(scp -v -o 'Compression no' -o 'StrictHostKeyChecking no' $test_file $server:$server_dir/$test_file 2>&1 | \
  grep "Bytes per second" | \
  sed "s/^[^0-9]*\([0-9.]*\)[^0-9]*\([0-9.]*\).*$/\1/g")
up_speed=$(echo "($up_speed/1000)" | bc)
echo "Upload speed:   $up_speed KBps on $server"

if [ "$do_download" == "1" ]; then
    # download test
    echo "Testing download from $server..."
    down_speed=$(scp -v -o 'Compression no' -o 'StrictHostKeyChecking no' $server:$server_dir/$test_file $test_file 2>&1 | \
      grep "Bytes per second" | \
      sed "s/^[^0-9]*\([0-9.]*\)[^0-9]*\([0-9.]*\).*$/\2/g")
    down_speed=$(echo "($down_speed/1000)" | bc)
    echo "Download speed: $down_speed KBps on $server"
fi

# clean up
echo "Removing test file, $server:$server_dir/$test_file..."
ssh $server "rm $server_dir/$test_file"
echo "Removing test file locally..."
rm $test_file