My God, it’s full of RSTs…

mygod

I had a bit of a debate with a network engineer last week. We’d taken a trace from a load balancer to try and understand why outlook anywhere sessions were getting blocked and terminated randomly (short answer; port exhaustion because there were insufficient IP addresses in the NAT pool on the server facing side of the load balancer, but that’s not important right now). I pointed out the reset packets coming from the load balancer. He countered with “Well why are there so many resets coming from the CLIENT? EH? EH? yeah… you heard me…” etc etc. They do go on, network engineers. And he’s right. Outlook does pump out a bunch of RST packets. On purpose. Allow this trace to illustrate – this is me opening a colleague’s calendar in Outlook, and then closing it.

outlook session temrinating with RST

So why does it do that? First, we need to talk about how TCP sessions are *supposed* to be terminated. The machine that is done talking sends a FIN/ACK, and goes into the FIN_WAIT_1 state until it receives an ACK to it’s FIN/ACK, at which time it goes into FIN_WAIT_2. The receiving machine sends an ACK, goes to the CLOSE_WAIT state, and then sends it’s own FIN/ACK. It will stay in the CLOSE_WAIT state until the application sends a close(). The first machine ACKs that FIN/ACK, and goes into the TIME_WAIT state for about 4 minutes (according to RFC793). This is supposedly to wait for any stray packets, but it’s actually because they are lazy.

tcp_ladder_states

So it’s possible to have the socket open but useless for minutes – one side waiting for the application to notice the socket is ready to close, the other waiting for a period equal to twice the maximum segment lifetime to expire (this is all explained MUCH more clearly here: http://j2eedebug.blogspot.co.uk/2008/12/difference-between-closewait-and.html).

time_wait

This is wasteful, to say the least, so many applications choose not to use FIN/ACK, but to issue a RST/ACK instead, which has the huge benefit of closing the socket down on both sides immediately, with none of that messy waiting around. This allows server ports to get recycled much faster. On the downside, it grinds network engineers’ gears. (is that a downside? really?)

this is documented (although not explicitly for Outlook) here:

https://support.microsoft.com/en-us/kb/272933

There’s a good Microsoft article on TCP states here:

https://support.microsoft.com/en-us/kb/137984

and advice on what you can do if your application insists on sending FINs, and you’re suffering from port exhaustion (it’s for SQL server, but you can’t have everything, right?):

https://support.microsoft.com/en-us/kb/328476

Advertisements
Post a comment or leave a trackback: Trackback URL.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: