Slow transfers - RFC 1323 scaling issue?

alanbryan · Jun 6, 2012

I think I may be having an issue with window scaling (RFC 1323) and am hoping that someone can enlighten me on what's going on.

Server: FreeBSD 9, apache22, serving a static 100MB zip file. 192.168.18.30
Client: Mac OS X 10.6, Firefox 192.168.17.47
Network: Only a switch between them - the subnet is 192.168.16/22
(In this test, I also have dummynet filtering simulating an 80ms ping time on all IP traffic. I've seen nearly identical traces with a "real" setup, with real internet traffic/latency also)

Questions:

Does this look normal?
Is packet #2 specifying a window size of 65535 and a scale of 512?
Is packet #5 then shrinking the window size so it can use the 512 scale and still keep the overall calculated window size near 64K?
Why is the window scale so high?

Here are the first 6 packets from wireshark. For packets 5 and 6 I've included the details showing the window size and scaling factor being used for the data transfer.

Code:

No. Time Source Destination Protocol Length Info

108 6.699922 192.168.17.47 192.168.18.30 TCP 78 49190 > http [SYN] Seq=0 Win=65535 Len=0 MSS=1460 WS=8 TSval=945617489 TSecr=0 SACK_PERM=1

115 6.781971 192.168.18.30 192.168.17.47 TCP 74 http > 49190 [SYN, ACK] Seq=0 Ack=1 Win=65535 Len=0 MSS=1460 WS=512 SACK_PERM=1 TSval=2617517338 TSecr=945617489

116 6.782218 192.168.17.47 192.168.18.30 TCP 66 49190 > http [ACK] Seq=1 Ack=1 Win=524280 Len=0 TSval=945617490 TSecr=2617517338

117 6.782220 192.168.17.47 192.168.18.30 HTTP 490 GET /utils/speedtest/large.file.zip HTTP/1.1

118 6.867070 192.168.18.30 192.168.17.47 TCP 375 [TCP segment of a reassembled PDU]

Details:

Transmission Control Protocol, Src Port: http (80), Dst Port: 49190 (49190), Seq: 1, Ack: 425, Len: 309
Source port: http (80)
Destination port: 49190 (49190)
[Stream index: 4]
Sequence number: 1 (relative sequence number)
[Next sequence number: 310 (relative sequence number)]
Acknowledgement number: 425 (relative ack number)
Header length: 32 bytes
Flags: 0x018 (PSH, ACK)
Window size value: 130
[Calculated window size: 66560]
[Window size scaling factor: 512]
Checksum: 0xd182 [validation disabled]
Options: (12 bytes)
No-Operation (NOP)
No-Operation (NOP)
Timestamps: TSval 2617517423, TSecr 945617490
[SEQ/ACK analysis]
TCP segment data (309 bytes)


119 6.867131 192.168.18.30 192.168.17.47 TCP 1514 [TCP segment of a reassembled PDU]

Details:

Transmission Control Protocol, Src Port: http (80), Dst Port: 49190 (49190), Seq: 310, Ack: 425, Len: 1448
Source port: http (80)
Destination port: 49190 (49190)
[Stream index: 4]
Sequence number: 310 (relative sequence number)
[Next sequence number: 1758 (relative sequence number)]
Acknowledgement number: 425 (relative ack number)
Header length: 32 bytes
Flags: 0x010 (ACK)
Window size value: 130
[Calculated window size: 66560]
[Window size scaling factor: 512]
Checksum: 0xe5cf [validation disabled]
Options: (12 bytes)
No-Operation (NOP)
No-Operation (NOP)
Timestamps: TSval 2617517423, TSecr 945617490
[SEQ/ACK analysis]
TCP segment data (1448 bytes)

Orum · Jun 18, 2012

alanbryan said:
Does this look normal?

Yes.

alanbryan said:
Is packet #2 specifying a window size of 65535 and a scale of 512?
Is packet #5 then shrinking the window size so it can use the 512 scale and still keep the overall calculated window size near 64K?

I assume by "packet #2" you mean number 115. And yes, you are correct, the window size is 64K with a scaling of 512. However, note that verbatim from RFC 1323, "The Window field in a SYN (i.e., a <SYN> or <SYN,ACK>) segment itself is never scaled."
As for "packet 5" (really packet 118 in this trace), yes, that appears to be the case. You may be able to change this behavior by tweaking net.inet.tcp.sendspace and net.inet.tcp.recvspace.

alanbryan said:
Why is the window scale so high?

Why not? The whole idea behind RFC 1323 is to increase the maximum window size at the (potential) expense of resolution, analogous to how floating point numbers are stored. Then consider the fact that it still needs to have some backwards compatibility with devices that don't support it (which is why it still makes use of the window size field to begin with), and that the window scale can only be negotiated in SYN segments ("This option is sent only in a SYN segment (a segment with the SYN bit on), hence the window scale is fixed in each direction when a connection is opened"), and it should be obvious why a large window scale value is chosen.

Edit: Oh, and if you want to play around with it disabled for testing purposes (I don't recommend you leave it disabled in normal use) just set net.inet.tcp.rfc1323=0

Slow transfers - RFC 1323 scaling issue?

alanbryan

Orum