HTTP

Hypertext Transfer Protocol · Application layer

HTTP is a text protocol: a request line, some headers, and optionally a body. The subtlety is that it rides on TCP's byte stream, so one HTTP message can span several packets and several messages can share one — a distinction that trips up nearly everyone reading their first capture.

Understand these first

TCP — Turns unreliable packets into a reliable, ordered byte stream.

Why does this protocol exist?: TCP delivers a stream of bytes with no structure at all. Two programs need an agreed way to say 'give me this thing' and 'here it is, and here is what it is'.
Which problem does it solve?: Request a resource and receive it, with enough metadata that the receiver knows where the response ends and what it contains.
Who participates?: A client and a server, often with proxies, caches and load balancers in between that are themselves both.
Which layer uses it?: Application layer, over TCP (port 80) or over TLS (port 443). These lessons teach HTTP/1.1 — HTTP/2 changes the framing and HTTP/3 changes the transport entirely.
What does it depend on?: TCP for a reliable ordered stream, DNS to find the server, and TLS when it is HTTPS.
What depends on it?: Essentially all web traffic, and most machine-to-machine APIs.
What state does each participant maintain?: None, by design — each request is self-contained. Applications add state on top with cookies and tokens. HTTP/1.1 does keep the TCP connection open between requests, which is connection reuse rather than protocol state.
Which fields matter?: request line — Method, path and version on one line. The version matters more than people expect — it changes framing rules.
Host header — Mandatory in HTTP/1.1. One IP address serves many sites, and this is the only thing distinguishing them.
Content-Length — Where the body ends. Without it (or chunked encoding) the receiver cannot tell a finished response from a stalled one.
Transfer-Encoding: chunked — The alternative when the length is not known in advance — each chunk states its own size.
status line — 2xx succeeded, 3xx go elsewhere, 4xx you were wrong, 5xx the server was wrong. The class is the first diagnosis.
What does normal behaviour look like?: Request headers, blank line, optional body; then response status, headers, blank line, body. On a persistent connection the next request follows immediately on the same TCP stream.
What does failure look like?: A 4xx or 5xx status, which is HTTP working correctly and reporting an application problem. Or a response that never completes, because Content-Length promised more bytes than arrived — which looks like a hang rather than an error.
How can I prove the diagnosis from packets?: Reassemble the stream rather than reading individual packets. A message may span several segments or share one with the next; the framing rules, not the packet boundaries, say where it ends.
How does it interact with the rest of the stack?: HTTP is where the byte-stream nature of TCP becomes visible and awkward: it must frame its own messages because TCP will not. Under TLS the headers are encrypted and only the timing and sizes remain visible.

Guided course — 3 lessons

Read in order. Every lesson pauses the capture on the packets it is talking about, and every one reads at three levels — so the time below is an estimate from the number of steps, not a measurement of anybody.

Troubleshooting labs — 1

A symptom, a capture, and no answer given. Each one is a lesson from the course above, listed again here because the diagnosis is worth coming back for.

When One Message Needs Many Packets

Find the delay and say whether the application had anything to do with it.

Reference material

The terms this module introduces, and the specifications behind them. Both lists come from the lessons themselves rather than from a list kept beside them.

HTTP

HTTP — end of module →

That you can separate what HTTP says from what carries it, and place the versions correctly.

Where HTTP sits in a real request

These captures run HTTP as one stage of a longer chain, and hand its result to whatever comes next.

Opening a website, end to end

You plug in a laptop and type a name. Four protocols run before a single byte of the page is requested, and each one exists because the one before it could not do this job. Watch what each stage hands to the next.

Opening a secure website

The same four stages, plus one: before any request is sent, the server has to prove it is the server. Note what a capture can still read after encryption starts — and what it cannot.

A routed enterprise application, end to end

The office journeys stop at the gateway. This one starts before it: two routing protocols fill one routing table, then ARP and DNS do their usual jobs, then an application opens a connection across the result. Nothing in the application's packets refers to a routing protocol — and that is what the routing protocols are for.

Control plane and data plane

Two captures of one network. In the first, the routers talk only to each other and no user traffic exists at all. In the second, user traffic crosses the network — while the routing protocols carry on saying hello in the background, because they never stop. Not one of their packets carries a byte of anybody's data. OSPF and BGP create forwarding information; the routing table selects a path; IP forwards packets; TCP, TLS and HTTP carry the application. Each of those depends on the one before it having already finished.

Diagnosis: the browser warns before showing anything

The connection opens fine, bytes flow both ways, and then the client hangs up on purpose. Something in what the server sent was unacceptable.

Diagnosis: the page loads, and it is an error page

Every layer beneath does its job perfectly. This is the case where the network is genuinely not at fault — and being able to prove that is the skill.