All right packet heads, welcome back to the channel! So in this video what we're gonna
do is take a closer look at retransmissions. You don't have to look far in a pcap to find
one. But how can we understand it and what do we do about it? Stick around.
All right so in this video we're going to take a closer look at tcp retransmissions.
No lie, as a packet analyst, I spend a lot of time taking a look at tcp retransmissions,
and no doubt you as you've looked through your pcaps, you've run into them. You're
running into different flavors of retransmissions, spurious retransmissions, fast retransmissions.
So I thought we could start with a basic tcp retransmission, dig into it, and we could
look at also what to do about it. What does that retransmission tell us about our flow of data and
the network that's carrying that flow? So go ahead and hit the link in the description down below to
download the pcap and you can follow right along. Let's get to it. So here in this trace file we can
see we have a simple tcp thread - SYN, SYN/ACK, ACK - this is just a single tcp conversation.
We can see that the client is sending out a client hello, going out and talking to
our server. Let me just adjust this a little bit - just going to blow this up for you guys
just a little bit, and i'm going to readjust my columns over here, my time column this is
set to - if we come to view time display format second since beginning of capture.
So this is basically my running total of time. Next to that I have my delta time. Now if you're
not sure how to set up that delta time you can click this video up here, where I go into this
basic setup of what you see here on my wireshark profile. Okay so it doesn't take us long to run
into problems, right. We see a lot of black lines with red letters - AH! Is our network falling??
What can we learn?? Why do we see all these tcp retransmissions?? Well let's analyze. So here we
can see in packet four that the client's trying to send this client hello over to the other side.
And this is where for me in this profile, something that I like to have is my tcp segment
length. You see I have 214 there. So instead of having the full frame length including the
ethernet frame, like seeing that full length up there which is the standard profile in wireshark,
I like actually to see how much payload is there. What that does is it shows me what packets are
just empty acknowledgments and which ones are actually carrying data. So to get that if you
just come down to tcp - pick any packet - and you come down here to tcp segment length, right click
you can apply as column to add that as a column. Again I just like to see that just because it's
easier to spot which packets are carrying data and which ones are empty ACKs. So here the client - we
can tell we're capturing client-side because of our handshake our SYN goes out, SYN/ACK comes
back and we're capturing on 101. That's where our delay was perceived. So our client hello goes out,
214 bytes, and here's the deal, we never get a response. We never get an acknowledgement.
We don't hear anything from the other side. So what does the client do? Well, when it sends
these bytes out - the tcp sender - what it does is it actually starts a stopwatch internal to itself.
It goes "You know what? I sent out those 214, I'm going to go ahead and start a timer,
and when that retransmission timer expires, I'm going to go ahead and resend that data." Well
how long is that retransmission timer? Well that depends upon the type of stack that we're running.
What some of its default settings are. But as a general rule of thumb, what you'll find with
a retransmission timer, it's typical to see it if you see the network round trip time and the
initial round trip time in the handshake. So since SYN/ACK is around 85 milliseconds,
it's not uncommon to see the amount of time for a retransmission timer to be
a multiple of that, at least three or four times the initial network round trip time.
What that does is it gives the data opportunity to get to the other side and get that acknowledgement
back. Uh what if we took a different path? What if the client on the other side was busy ACKing some
other data, or some on some other connection and then it finally gets around to us? So it
does give some leeway time there before we just fire off that retransmission. But as we can see
here our RTO or retransmission timeout right there, in this case it's 307 milliseconds.
So there's our first retransmission. Well after this around, 90 milliseconds later,
what we see is that the other side goes ahead and sends us an acknowledgement.
Now that number is around the network round trip time. So 90 milliseconds here I can see 85
milliseconds up here. So that was about the time I would expect to hear an acknowledgement. What
this does is it tells us that our data got there, but we also see some other information here. We
notice tcp previous segment not captured. This is interesting. So if I take a look at that this, is
the other side sending back to me, if I come down here and I take a look at the sequence numbers.
So i'm going to come down to sequence number. Well the sequence number on this ingress ACK
jumped. The last time that I heard data coming in from the other side, the sequence number,
if I go back to packet 2, it's the last time I saw something from this direction. The sequence number
advanced from 0 to 1 in the handshake, so the very next packet that I should have seen coming in from
that other side should have started at one, but it doesn't. Instead we see it jump up to 4067,
and that's why wireshark is flagging - whoa whoa tcp previous segment not captured - the sender
on the other side thinks that it was able to advance the sequence number up to 4067.
He's like hey, I sent you 4067 bytes or 4066 bytes that's already behind me,
I've moved forward in my sequence numbers. This is why we see tcp previous segment not captured.
Wireshark's going whoa wait a second, we saw a jump in sequence numbers. Now since the other side
advertised that it was at 4067, but it backs up and resends previous data, that's why wireshark
goes - oh this is a retransmitted segment. Now there's a few interesting things that
are going on here. First of all this is what my eyes do. Two seconds hmm. Also low
segment size 536. In fact we can look into the rfcs and take a look a deeper look into tcp,
but basically this is basically the tcp minimum mss that it wants to try. It doesn't want to go
lower than 536. This is basically the min. That's why my eye catches it. This is a tcp
stack that's saying you know what let me just give this a shot to see if this works.
And sure enough it does. This packet makes it through and the client is able to acknowledge
it. So what happened here? Well likely just by studying these retransmissions,
what we see is that the other side tried to get data across, but the mss ran into problems.
Likely see it was thinking okay, I got a larger mss right, this receiver can handle 1460 why
don't i go ahead and send segments of 1460 across to the other side. Well it tried to send those.
Those probably got dropped on the other side. It probably tried to retransmit. Retransmit.
Retransmit. And some mtu problem happened in the middle and finally the other side says - you know
what before i give up on this and just reset this connection, let me just try the minimum mss that
i'm allowed to. Let's just try 536 before we give up. And sure enough that packet made it through
so then we acknowledge and then the other side goes - oh wow that packet made it through,
great! Okay here's another packet that was missing, here's another one that was missing,
client says sweet we're good we're talking. Here's another packet that was missing. So what does this
tell us? Well first of all, our client hello likely made it across the other side. This very
first one but the problem is is that the sender on the other side likely tried a larger segment size
than the network can handle. There was probably an MTU that it ran into. Now this would be a very
interesting trace to take from the opposite side. This is oftentimes why I like to do dual side
captures, one in the client, one on the server, and then we can see the server let go of 1460s or
maybe even 1440s and then we get that feedback from the network, maybe an icmp - I wanted to
send this on but the do not fragment bit was set, so I had to drop it - maybe that's coming from a
router and that would tell me what link is having the mtu the lower mtu than i'm trying to use. So
this is just one form of retransmission we hear. We saw the RTO expire, we saw a retransmission,
and we also saw retransmissions coming from the other side that showed us about a gap in
data that likely was due to an mtu problem. A lot more to say about retransmissions and I'm going
to make sure to cover those topics in other videos - spurious retransmissions, fast retransmissions,
and even more. So thanks for stopping by the channel, make sure to like, subscribe,
forward, share, all the things to your friends and i appreciate you viewing! Take care!