TCP in Production: Retransmissions, Keepalive, Window Tuning & Debugging

01 · Performance

TCP Performance

TCP throughput is fundamentally bounded by the relationship between window size and round-trip time.

The bandwidth-delay product (BDP = bandwidth × RTT) tells you how many bytes can be 'in flight' at once to fully utilize a link. If the TCP window is smaller than the BDP, the connection is window-limited and can never reach the link's full bandwidth, no matter how fast the network actually is.

Worked example: why a 64 KB window caps a fast link

Illustrative numbers: a 100 Mbit/s path with an 80 ms RTT. BDP = 100,000,000 bit/s × 0.080 s = 8,000,000 bits = 1,000,000 bytes (~1 MB) must be in flight to keep the pipe full. With only the original 65,535-byte window (no window scaling), throughput is capped at window / RTT = 65,535 B / 0.080 s ≈ 819 KB/s ≈ 6.6 Mbit/s — about 6.6% of the link. The same arithmetic at 1 Gbit/s × 80 ms gives a BDP of 10 MB, so the receive buffer and window scaling must grow with the link, not just the NIC speed.

02 · Latency

Latency

Every new TCP connection pays at least one full round trip before any application data can be sent — and that's before TLS.

The three-way handshake alone costs about 1 RTT before normal application data flows. Add TLS 1.2's handshake (another 1-2 RTTs) and you can be 2-3 RTTs deep before the first useful byte moves. TLS 1.3 reduces that cost, and TCP Fast Open can help in some deployments, but connection reuse is still the mainstream fix.

03 · Reliability under stress

Packet Loss

Packet loss doesn't just delay data — it can catastrophically reduce throughput, because TCP interprets loss as a congestion signal.

Even 1-2% packet loss on a high-bandwidth, high-latency link can cut achievable throughput dramatically (the Mathis et al. approximation puts steady-state throughput near (MSS / RTT) × 1.22 / √p for loss rate p), because congestion control halves the congestion window on every detected loss event, and it takes many round trips to climb back up.

Plugging the same illustrative path into the Mathis approximation (MSS 1,460 B, RTT 80 ms): at 1% loss, throughput ≈ (1,460 / 0.080) × 1.22 / √0.01 ≈ 223 KB/s ≈ 1.8 Mbit/s; at 0.01% loss it is ten times higher, ≈ 17.8 Mbit/s. The model is an approximation for loss-based Reno-style congestion control, not a prediction for CUBIC/BBR on a specific path, but it shows why even rare loss dominates on long paths.

04 · Recovery mechanics

TCP Retransmissions

TCP has two independent ways to detect loss and resend the missing data — a slow, conservative one (timeout) and a fast, opportunistic one (duplicate ACKs).

Retransmission Timeout (RTO)

If no ACK arrives within the retransmission timeout, the segment is resent, cwnd resets aggressively (often to 1), and RTO itself doubles (exponential backoff) for subsequent retries.

Fast Retransmit

Three duplicate ACKs for the same byte trigger an immediate retransmission of the missing segment — without waiting for a timeout, and with a much gentler congestion window reduction (Fast Recovery).

05 · RTO tuning

Timeouts

The retransmission timeout is not a fixed constant — it's computed dynamically from measured round-trip times.

TCP maintains a smoothed RTT estimate (SRTT) and RTT variance, and computes RTO as roughly SRTT + 4 × RTTVAR (Jacobson/Karels algorithm). A network with highly variable latency (e.g. mobile, Wi-Fi) produces a larger, more conservative RTO.

06 · Throughput tuning

Window Tuning

The default OS TCP window sizes are often too small for modern high-bandwidth, high-latency links.

Window scaling (a TCP option defined in RFC 7323, which obsoletes RFC 1323, negotiated during the handshake) extends the advertised receive-window field far beyond the original 16-bit 64KB limit. That matters for flow control on long, fast links; the sender's actual in-flight data is still capped by min(rwnd, cwnd), so scaling the receive window does not bypass congestion control.

07 · Idle connections

TCP Keep Alive

TCP keep-alive is a low-level probe mechanism for detecting dead peers on otherwise-idle connections — distinct from application-level (e.g. HTTP) keep-alive.

Without keep-alive, an idle TCP connection whose peer has silently disappeared (crashed, network partition, NAT timeout) can sit open indefinitely — the local OS has no way to know the peer is gone until it tries to send data.

08 · Observability

Monitoring

TCP-level metrics reveal problems that application logs alone never will.

Retransmission rate

A rising retransmit percentage is often the earliest signal of network degradation, well before application-level error rates move.

TIME_WAIT / connection counts

Tracking socket states (via tools like ss -s or netstat) surfaces connection churn and potential port-exhaustion risk before it becomes an outage.

RTT distribution

p50/p95/p99 RTT trends distinguish 'a few slow clients' from 'the whole network path degraded.'

09 · Failure modes

Common Production Issues

Most 'mysterious' production networking issues turn out to be well-known TCP behaviors once you know what to look for.

Port exhaustion

Under high connection churn, a host can run out of ephemeral source ports, especially when combined with lingering TIME_WAIT sockets.

Head-of-line blocking

Because TCP delivers bytes to the application in order, one lost segment can stall delivery of everything behind it to that socket until the gap is filled — even if later segments already arrived. That transport-level head-of-line blocking is a key motivation for HTTP/3's move to QUIC.

Nagle's algorithm + delayed ACK interaction

The classic latency trap is not just 'two batching features exist' — it's that if a sender has one small unacknowledged segment outstanding, Nagle's algorithm may hold the next tiny write until an ACK arrives, while the peer's delayed-ACK timer may intentionally wait before sending that ACK. The result can be tens to hundreds of milliseconds of avoidable latency on request/response workloads; see Nagle's Algorithm for the protocol mechanics.

10 · Diagnosis

Debugging

A structured debugging approach beats guesswork every time — narrow down layer by layer.

Packet captures

tcpdump/Wireshark reveal the ground truth — actual segments, flags, sequence numbers, and retransmissions on the wire, when application logs are ambiguous.

Connection state

ss -tin (Linux) exposes live congestion window, RTT estimate, and retransmit counters per socket — often enough to diagnose a throughput problem without a capture.

textIllustrative `ss -tin` output for one socket (constructed for teaching, not captured from a real system)

ESTAB 0 0 10.0.0.5:51514 203.0.113.10:443
   cubic wscale:7,7 rto:204 rtt:80.5/4 mss:1448 cwnd:44 ssthresh:44
   bytes_acked:812000 retrans:0/3 send 6.33Mbps delivery_rate 6.2Mbps

How to read it: rtt:80.5/4 is smoothed RTT / variance in ms; cwnd:44 is in segments, so in-flight data is capped near 44 × 1,448 B ≈ 64 KB, and send ≈ cwnd × mss / rtt ≈ 6.3 Mbit/s. A path that can carry 100 Mbit/s but sits at ~6 Mbit/s with retrans:0/3 (zero in recovery, 3 retransmits total) points at a window limit (or small buffers) rather than loss. Rising retrans and a shrinking cwnd instead point at loss or congestion; a small peer window shows up as a receive-window limit.

Key takeaways

Throughput is bounded by bandwidth-delay product — window size must scale with both bandwidth and RTT to saturate a link.
Loss triggers congestion control's aggressive backoff, which is why even small loss rates devastate throughput on long, fast links.
Fast retransmit (dup ACKs) recovers loss far faster than waiting for RTO — but only when more data is already in flight.
TCP keep-alive and application heartbeats solve different problems and are often both required in front of load balancers/NAT.
Most 'mystery' latency bugs (Nagle/delayed-ACK interaction, port exhaustion, head-of-line blocking) are well-documented, well-understood TCP behaviors.