Tuesday, September 06, 2011

Protocol & network speed

Greetings, Internet audience.

Out of idle curiosity, and a plaguing question from a client about why he wasn't getting as much throughput as he expected on super-sweet gigabit Catalyst 4948 switches as he expected, meaning he expected ~600Mbps throughput, and got about 130Mpbs.

I suspected two things:

1) bandwidth limitations on the servers, meaning that a 1Gbps NIC doesn't get 1Gbps speeds on the internal BUS, and thus, won't get near 1Gbps wire speed.
2) protocol. We were testing speed using scp, and I suspected encryption was reducing the speed.

Since #2 is a bit easier for me to prove, I staged a test on my home office network, at a time I was the only one on the network.

My plan:
copy a 706330624 byte file locally, using three protocols:
scp (tcp/22), tftp (udp/69), and ftp (tcp/21)
my home network is comprised of inexpensive 10/100Mbps Netgear switches, the kind you can buy at Fry's for $30, or could a few years back, you get the point.

Note the results are in Mega "bits" per second, not Mega "bytes" per second. For MBPS, divide by 1024, or check out this handy calculator-page:
http://www.matisse.net/bitcalc/

Results:
Protocol Mbps
scp ~20Mbps
tftp ~6Mbps
ftp ~89Mbps

As you may have already figured out, the cost of encryption on network speed, is pretty high, and if you are hearing complaints about your network speed, make sure and test with FTP for the lower-protocol-overhead results. My swag as to why tftp is so slow is udp retries, but this is slower than I expected. If anyone has an opinion to add, comment away.

2 comments:

James said...

I believe the reason the TFTP protocol is so slow because the next packet will not be sent before the previous packet's ACK is received.

Yanis said...

FTP is the best quick and dirty test of network speed point-to-point. Since the protocol will adjust it's RWIN dynamically to get the best possible performance from current network conditions.

I always use that and a copy operation from Windows to Windows using Explorer. You gotta love the ETA calculation MS uses though... :)