Slow Download Speeds from AWS S3

S

sydney6

Hi there,

I'm seeing continuous slow download speeds from Amazon S3, but only on FreeBSD. Other OSes saturate the connection without problems.

This happens with 10.3-RELEASE and 11.0-RELEASE and only with AWS S3 in different regions (Ireland, London, Frankfurt, US Standard) whilst using fetch, curl, et. al.

I have tested this on multiple machines, bare metal, bhyve, Xen and VMware VMs, routed setups and direct pppoe links.. all the same.

Anyone seeing similar issues?
 
Here's a url to try:

http://s3.eu-central-1.amazonaws.com/4f48caf1d8bcbef8/c5b38f8b3625d2b6/zerofile.raw

Also, when doing double NAT, i.e. VMware Fusion FreeBSD Guest with "Share with my Mac"-Interface, the machine is doing completely fine, as in saturates the link, 90Mbps otherwise between 12-15Mbps..

I also switched the FreeBSD Routers with Linux-based ones and with the isp-provided router box, with the same result.

Edit: I have launched VMs with Digital Ocean to "rule out" my ISP and there seems to be the same issue. Downloading from S3 is multiple times slower than any other connection to services outside of S3 or on any other OS.
 
I haven't had much time to look at this, but the link you provided downloads a lot slower on my FreeBSD 11.0 droplet than the 100MB speedtest file.

Code: 
curl -o /dev/null "http://s3.eu-central-1.amazonaws.com/4f48caf1d8bcbef8/c5b38f8b3625d2b6/zerofile.raw"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  100M  100  100M    0     0  3715k      0  0:00:27  0:00:27 --:--:-- 3757k
Code: 
curl -o /dev/null "http://speedtest.wdc01.softlayer.com/downloads/test100.zip"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  100M  100  100M    0     0  10.2M      0  0:00:09  0:00:09 --:--:-- 16.8M
 
Hi,

thanks for your results..

FYI, i have also sent this to the freebsd-net mailing list, in case someone would like to follow up on the issue.

If i have news on this one, i will update this forum post.
 
sydney6 said:
Hello,

I took this issue to the -net mailing list and thanks to Steve Hartland the issue is resolved.

https://lists.freebsd.org/pipermail/freebsd-net/2017-February/047234.html

The problem was/is (probably) lacking TCP timestamps in AWS S3 traffic resulting in a failure of FreeBSDs TCP window scaling mechanism to increase the TCP receive buffer size and therefore resulting in lower troughput over higher latency links.
Click to expand...

Very interesting - thanks for posting the outcome.
 
You must log in or register to reply here.
Back
Top