The Engineer's Exchange

Why layers

Getting a message from a program on your laptop to a program in a data centre on another continent is an enormous problem, so the internet splits it into independent sub-problems and stacks the solutions. The application layer decides what to say (HTTP, the request for a page). The transport layer gets that message to the right program on the far host and, with TCP, makes sure it arrives complete and in order. The network layer (IP) gets a packet to the right host, hop by hop, across any number of networks. The link layer moves bits across a single stretch of wire or radio to the next device. Each layer is a contract: it offers a service to the layer above and relies on the one below, so you can reason about HTTP without thinking about voltages, and swap Wi-Fi for Ethernet without changing a line of your app.

This is the TCP/IP model, four layers. You'll also hear the older seven-layer OSI model, which is mostly useful now as vocabulary: when someone says "a layer 7 load balancer" they mean it understands HTTP, and "layer 4" means it works at TCP. We'll use the four-layer model and borrow the numbers where they're handy.

Encapsulation: envelopes inside envelopes

A message doesn't move down the stack unchanged; each layer wraps it in a header of its own, like putting a letter in an envelope, then that envelope in a bigger one. On the far side the process reverses, each layer reads and strips its own header and hands the rest up. Here is the wrapping, with the real header sizes:

from dataclasses import dataclass

# Each layer wraps the one above in its own header, the way envelopes nest inside envelopes.
# These are the real header sizes for a classic Ethernet / IPv4 / TCP / HTTP stack.
LAYERS = [
    ("application", "HTTP", 0,  "what: the request/response you wrote"),
    ("transport",   "TCP",  20, "which program: ports, plus reliability (seq, ack)"),
    ("network",     "IPv4", 20, "which host: source and destination IP addresses"),
    ("link",        "Ethernet", 14, "which next hop: MAC addresses, one wire-segment"),
]


def encapsulate(payload_bytes):
    """Send side: each layer prepends its header. Returns the frame size at each step."""
    size = payload_bytes
    steps = []
    for layer, proto, header, _ in LAYERS:
        size += header
        steps.append((layer, proto, header, size))
    return steps


def overhead(payload_bytes):
    total_header = sum(h for _, _, h, _ in LAYERS)
    on_wire = payload_bytes + total_header
    return total_header, on_wire, total_header / on_wire


if __name__ == "__main__":
    print("a layer only reads its own header, then hands the rest up or down:")
    for layer, proto, header, why in LAYERS:
        print(f"  {layer:12} {proto:9} +{header:2}B  {why}")

    print("\nencapsulating a 1-byte payload (e.g. a single keystroke):")
    for layer, proto, header, size in encapsulate(1):
        print(f"  after {layer:12} ({proto}): {size:3}B on the wire")
    h, wire, frac = overhead(1)
    print(f"  -> {h}B of headers to carry 1B: {frac:.0%} overhead (why tiny packets waste the network)")

    h, wire, frac = overhead(1460)   # a full TCP segment (1500 MTU - 40 bytes IP+TCP)
    print(f"a full 1460B segment: {h}B headers, {wire}B on the wire, {frac:.1%} overhead")

Two lessons fall out of the numbers. A layer reads only its own header: a router looks at the IP header to decide where to forward the packet and never opens the TCP or HTTP inside, which is both an efficiency and a privacy property (and why TLS, chapter 6, can hide everything above IP). And headers have a fixed cost, 54 bytes here, so carrying a single byte is 98% overhead while a full 1460-byte segment is only 3.6%: this is why chatty, tiny messages waste the network and why batching helps, a theme that returns in every later chapter.

The journey of a request

Follow one HTTP request. Your code hands HTTP to the transport layer, which splits it into segments and adds a TCP header with source and destination ports (so the far host knows which program the data is for) and sequence numbers (so it can be reassembled in order). Each segment goes to IP, which wraps it in a packet with source and destination IP addresses and hands it to the link layer, which frames it for the first hop. Then it travels: across your Wi-Fi to your router, which forwards it toward its destination; through your ISP; across the internet's backbone, router to router, each one reading the IP header, deciding the next hop, and passing it on; until it reaches the server's network, its host, and finally, unwrapped layer by layer, the server program. The reply makes the same trip in reverse. The whole round trip is what chapter 16 teaches you to count, because its duration is set by physics, not by your code.

The ideas that shaped it

• Packet switching. The internet doesn't reserve a dedicated line for your conversation the way old telephone networks did; it chops data into packets that share links with everyone else's, each routed independently. That's why the internet is cheap and resilient (a packet routes around a failure) and also why latency varies (your packet waits in queues behind others).
• The end-to-end principle. Keep the network in the middle simple and fast, and put the intelligence, reliability, ordering, encryption, at the endpoints. It's why IP itself makes no promises (a packet may be lost, duplicated or reordered) and TCP builds reliability on top, at the ends. This principle explains more design decisions in this volume than any other.
• Addresses at every layer. A thing has a different name at each layer: a human-friendly hostname at the application layer (resolved by DNS, chapter 15), an IP address at the network layer (chapter 13), a port at the transport layer, and a hardware MAC address at the link layer. Confusing which name lives where is the source of many networking muddles.

Three inventions

Tim Berners-Lee's web rests on three things that still define it: a way to name any resource (the URL), a way to fetch it (HTTP, chapter 3), and a way to link resources into a web (hypertext). This chapter is about the first, because the name comes before the fetch, and because a surprising amount of networking and security is really about reading a URL correctly. The model is deliberately simple: a client asks for the resource at a URL, a server returns a representation of it, and that's a complete interaction. The resource might be a file, a database row rendered as JSON, or something computed on the spot; the URL names it without saying how it's produced.

Anatomy of a URL

from urllib.parse import urlsplit, urljoin, parse_qs, urlencode

DEFAULT_PORTS = {"http": 80, "https": 443, "ws": 80, "wss": 443}


def anatomy(url):
    u = urlsplit(url)
    return {
        "scheme": u.scheme,                       # how to talk: https, ws, mailto...
        "host": u.hostname,                       # which machine (a name, resolved by DNS)
        "port": u.port or DEFAULT_PORTS.get(u.scheme),  # which program; default by scheme
        "path": u.path,                           # which resource on that machine
        "query": dict(parse_qs(u.query)),         # parameters
        "fragment": u.fragment,                   # a spot within the resource; never sent to the server
    }


def origin(url):
    """The security boundary of the web: scheme + host + port. Same-origin or not."""
    u = urlsplit(url)
    port = u.port or DEFAULT_PORTS.get(u.scheme)
    return f"{u.scheme}://{u.hostname}:{port}"


if __name__ == "__main__":
    url = "https://user@shop.example.com:8443/products/42?color=blue&size=M#reviews"
    for k, v in anatomy(url).items():
        print(f"  {k:9}: {v}")

    print("\norigins (the web's security boundary):")
    for a, b in [("https://app.example.com/a", "https://app.example.com/b"),
                 ("https://app.example.com", "http://app.example.com"),      # scheme differs
                 ("https://app.example.com", "https://api.example.com")]:    # host differs
        print(f"  {origin(a)} vs {origin(b)}: {'SAME' if origin(a) == origin(b) else 'different'} origin")

    print("\nresolving relative URLs against a base (how links in a page work):")
    base = "https://docs.example.com/guide/intro.html"
    for rel in ["./setup.html", "../api/ref.html", "/", "//cdn.example.com/x.js", "?v=2"]:
        print(f"  {rel:24} -> {urljoin(base, rel)}")

    print("\npercent-encoding keeps special characters out of the structure:")
    print("  " + "?" + urlencode({"q": "cats & dogs", "page": "2"}))

Each part answers a different question, and they nest from general to specific:

A few parts repay attention. The fragment (#reviews) is handled entirely by the browser and never sent to the server, which is why single-page apps once used it for routing and why you can't read it in server logs. The query carries parameters, but it's part of the URL, so it ends up in logs, history and referrer headers, never put secrets there (chapter 6 of The Engineer's Citadel returns to this). And the term URI you'll see in specs is the broader category; for practical purposes a URL is a URI that tells you how to locate the thing, which is the only kind you'll use.

Origin: the web's security boundary

The three parts scheme, host and port together form the origin, and the origin is the single most important concept the web uses for security. The browser's same-origin policy keeps one site from reading another's data based on it; cookies, CORS and much else are defined in terms of it (all covered in the security stage). The example shows the rule's sharp edges: https and http on the same host are different origins, because the scheme differs, and app.example.com and api.example.com are different origins, because the host differs, even though they're the same site to a human. Getting "same origin" right is getting the scheme, host and port all equal.

Relative URLs and links

Most URLs in a page aren't written out in full; they're relative to the page's address, and the browser resolves them against a base. The example shows the rules: ./setup.html replaces the last path segment, ../api/ref.html climbs a directory, a leading / is absolute from the host's root, and a leading // is protocol-relative, keeping the current scheme but changing the host (handy for CDNs, and a classic open-redirect trap). Relative URLs are what make a site movable between domains without rewriting every link, and understanding resolution is what stops the mysterious "why did this link go there" bug.

Percent-encoding and schemes

A URL has structure, /, ?, &, # all mean something, so any data you put inside it that contains those characters must be percent-encoded (%26 for &, and so on), exactly as the example encodes "cats & dogs". This is the URL's version of the separate-data-from-structure rule you'll meet everywhere in security: encode data so it can't be mistaken for structure. Beyond http and https, the scheme can select other protocols entirely, ws and wss for WebSockets (chapter 7), mailto:, file:, data:, so the scheme is the first thing to check when a URL behaves unexpectedly.

Text on the wire

An HTTP/1.1 request is simple enough to read and write by hand, which is exactly what this stage's deliverable asks you to do. It's a request line (method, target, version), a set of headers, a blank line, and an optional body. The response has the same shape with a status line instead. Parsing and building it is a few lines:

# HTTP/1.1 is text on the wire: a request line, headers, a blank line, then an optional body.
# This is the heart of the stage's deliverable: parse it yourself, no framework.
def parse_request(raw: bytes):
    head, _, body = raw.partition(b"\r\n\r\n")
    lines = head.split(b"\r\n")
    method, target, version = lines[0].decode().split(" ")
    headers = {}
    for line in lines[1:]:
        name, _, value = line.partition(b":")
        headers[name.decode().strip().lower()] = value.decode().strip()   # names are case-insensitive
    return {"method": method, "target": target, "version": version,
            "headers": headers, "body": body}


def build_response(status, reason, headers, body=b""):
    lines = [f"HTTP/1.1 {status} {reason}"]
    headers = {**headers, "Content-Length": str(len(body))}   # frame the body by its length
    lines += [f"{k}: {v}" for k, v in headers.items()]
    return ("\r\n".join(lines) + "\r\n\r\n").encode() + body


def decode_chunked(stream: bytes) -> bytes:
    """Chunked transfer-encoding: send a body whose length you don't know up front,
    as a series of (hex size, CRLF, bytes, CRLF) chunks ending with a zero-size chunk."""
    out, i = bytearray(), 0
    while True:
        nl = stream.index(b"\r\n", i)
        size = int(stream[i:nl].split(b";")[0], 16)   # hex size, optional ;extension
        if size == 0:
            break
        start = nl + 2
        out += stream[start:start + size]
        i = start + size + 2                           # skip the chunk and its trailing CRLF
    return bytes(out)


if __name__ == "__main__":
    raw = (b"GET /products/42 HTTP/1.1\r\n"
           b"Host: shop.example.com\r\n"
           b"Accept: application/json\r\n"
           b"Connection: keep-alive\r\n"
           b"\r\n")
    req = parse_request(raw)
    print("method:", req["method"], "| target:", req["target"])
    print("host header:", req["headers"]["host"], "| keep-alive:", req["headers"]["connection"])

    resp = build_response(200, "OK", {"Content-Type": "application/json"}, b'{"id":42}')
    print("\nresponse on the wire:")
    print(resp.decode().replace("\r\n", "\\r\\n\n"))

    chunked = b"9\r\n{\"id\":42}\r\n7\r\n,\"ok\":1\r\n0\r\n\r\n"
    print("decoded chunked body:", decode_chunked(chunked).decode())

That small file is most of an HTTP server. The rest of this chapter is what the fields mean and the one thing the parser must never get wrong: framing.

Methods: what you want done

The method states the intent. The ones that matter, and two properties that govern them:

Safe means it only reads, never changes state, so a crawler or a browser prefetch can call it freely (which is why putting a delete behind a GET is a classic bug). Idempotent means doing it twice has the same effect as once, which is what lets a client safely retry a timed-out PUT or DELETE but not a POST. These properties aren't enforced by HTTP; they're promises you keep when you implement the method, and the whole web (caches, retries, prefetching) assumes you keep them. The Engineer's Shipyard builds API design on top of this.

Status codes: how it went

The three-digit code's first digit is the category, and knowing the ranges is most of the job: 1xx informational (rare), 2xx success, 3xx redirection ("look elsewhere"), 4xx you (the client) got it wrong, 5xx I (the server) got it wrong. The ones worth memorising: 200 OK, 201 Created, 204 No Content; 301/308 permanent and 302/307 temporary redirects, 304 Not Modified (your cached copy is still good, chapter 4); 400 bad request, 401 unauthenticated, 403 forbidden, 404 not found, 409 conflict, 429 too many requests; 500 internal error, 502 bad gateway, 503 unavailable, 504 gateway timeout. The 4xx-versus-5xx line is the one that matters operationally: a spike in 4xx is usually a client or a bad deploy; a spike in 5xx is you.

Framing: where does the message end?

This is the question HTTP exists to answer on a byte stream, and the one your server must get exactly right. TCP gives you a stream of bytes with no message boundaries (chapter 9), so HTTP has to say where each message's body stops. There are two ways. Content-Length states the body's size up front, and the receiver reads exactly that many bytes, what build_response does. But sometimes you don't know the size when you start sending (a response generated on the fly), so chunked transfer-encoding sends the body as a series of sized chunks, each prefixed with its length in hex, ending with a zero-length chunk, which decode_chunked reassembles. Every HTTP/1.1 body uses one or the other. Getting framing wrong isn't just a bug; when a proxy and a server disagree about where a message ends, you get request smuggling, one of the nastier web vulnerabilities (The Engineer's Citadel, chapter 6), which is why the parser's handling of Content-Length and chunked has to be strict.

Keep-alive: reuse the connection

Opening a TCP connection costs a round trip (chapter 9), and HTTPS adds another for TLS (chapter 6), so opening a fresh connection for every request, as HTTP/1.0 did, is slow. HTTP/1.1 made connections persistent by default: after a response, the connection stays open (Connection: keep-alive) and the next request reuses it, paying the handshake once.

One subtlety the diagram flags: even with keep-alive, HTTP/1.1 sends requests serially on a connection, the next can't start until the previous response finishes (an attempt to overlap them, called pipelining, failed in practice because of head-of-line blocking). Browsers worked around it by opening six connections per host; HTTP/2 (chapter 5) solves it properly.

One more essential header: Host

HTTP/1.1 made the Host header mandatory, and it's why the modern web works: one server at one IP address can host thousands of sites, and the Host header tells it which one you want (virtual hosting). Without it, the IP address alone couldn't distinguish a.example from b.example on the same machine. It's also the header to be careful with on the server side, since trusting it blindly causes the cache-poisoning and password-reset bugs from The Engineer's Citadel.

Stateless by design

HTTP keeps no memory between requests: the server handles each one with no idea what you asked a moment ago. That sounds like a limitation, and it's actually the web's superpower, because any server can handle any request, so you can put a thousand servers behind a load balancer and it just works (chapter 18). The cost is that anything that should persist, who you are, what's in your cache, has to be carried explicitly, which is what this chapter's three mechanisms do.

Caching: the biggest lever there is

The fastest request is the one you never send. HTTP caching lets a response be reused, by the browser, a CDN, or a proxy in between, instead of fetched again, and getting it right is the single biggest performance win on the web. Two ideas drive it. Freshness: a response can carry Cache-Control: max-age=600, meaning "reuse me for 600 seconds without asking". Validation: when a cached copy goes stale, the cache doesn't blindly refetch; it asks "is my copy still good?" with the ETag it was given, and the server answers 304 Not Modified (keep it, a tiny response with no body) or sends a fresh 200. Here is the whole decision:

# An HTTP cache (browser, CDN or proxy) decides per request: serve a fresh copy, revalidate
# a stale one cheaply, or fetch anew. The rules come from Cache-Control and ETag headers.
class Origin:
    """A stand-in web server. The resource changes only when we say so."""
    def __init__(self):
        self.body, self.etag, self.max_age = b"v1 body", '"v1"', 10
    def fetch(self, if_none_match=None):
        if if_none_match == self.etag:                      # client's copy is still current
            return 304, None, self.etag, self.max_age        # tiny response, no body
        return 200, self.body, self.etag, self.max_age


class Cache:
    def __init__(self):
        self.store = {}     # url -> (body, etag, stored_at, max_age)

    def get(self, url, origin, now):
        entry = self.store.get(url)
        if entry is None:
            status, body, etag, max_age = origin.fetch()
            self.store[url] = (body, etag, now, max_age)
            return "MISS -> fetched", body
        body, etag, stored_at, max_age = entry
        if now - stored_at < max_age:
            return "HIT (fresh)", body                       # no network at all
        # Stale: revalidate cheaply with the ETag we hold.
        status, new_body, new_etag, new_max_age = origin.fetch(if_none_match=etag)
        if status == 304:
            self.store[url] = (body, etag, now, new_max_age)  # still good; refresh freshness
            return "REVALIDATE -> 304 (kept)", body
        self.store[url] = (new_body, new_etag, now, new_max_age)
        return "REVALIDATE -> 200 (replaced)", new_body


if __name__ == "__main__":
    cache, origin, url = Cache(), Origin(), "https://shop.example/logo.png"
    for now in (0, 5, 12, 20):
        label, body = cache.get(url, origin, now)
        print(f"t={now:2}s  {label:28} body={body.decode()}")
    print("\nnow the resource changes on the origin:")
    origin.body, origin.etag = b"v2 body", '"v2"'
    label, body = cache.get(url, origin, now=35)              # stale -> revalidate -> 200
    print(f"t=35s  {label:28} body={body.decode()}")

The trace shows it: a miss fetches and stores; within max-age it's a pure hit with no network; once stale it revalidates with the ETag and gets a 304, so it keeps the bytes it already had and just refreshes the freshness clock; and when the resource actually changes, revalidation returns 200 with the new body. That 304 is the quiet workhorse of the web: it turns "re-download the image" into "a few bytes confirming the image is unchanged".

A few more knobs matter in practice. no-store forbids caching entirely (for private, per-user data); no-cache confusingly means "cache it but always revalidate before use"; private lets the browser cache but not a shared CDN; s-maxage sets a separate lifetime for shared caches. And the Vary header tells caches which request headers change the response (Vary: Accept-Encoding), so they don't hand a gzipped copy to a client that can't decompress it. The pattern that makes static assets both cacheable forever and instantly updatable is to put a content hash in the filename (app.9f3c.js) and serve it with a one-year max-age: the URL changes when the content does.

Cookies: memory for a stateless protocol

To recognise you across requests, the server sends Set-Cookie: session=abc123 with a response, the browser stores it, and sends Cookie: session=abc123 on every subsequent request to that site. That round trip is how a stateless protocol gets state: the cookie is a little token the client carries back each time, and the server looks up who it belongs to. Cookies carry attributes that control their safety and scope, Secure (HTTPS only), HttpOnly (hidden from JavaScript), SameSite (not sent on cross-site requests), Domain and Path (scope), and an expiry. The security of these is a whole topic in its own right (The Engineer's Citadel, chapter 5); here the point is the mechanism: state on the web is something the client carries, not something the connection remembers.

Content negotiation: one URL, many representations

The same resource can come in different forms, and the client says which it prefers through Accept headers, while the server picks and announces its choice with Content-Type. The main axes: type (Accept: application/json vs text/html), language (Accept-Language: fr), and encoding (Accept-Encoding: gzip, br, zstd). That last one is quietly one of the web's biggest wins: the client advertises which compression it understands, the server compresses the response, and text payloads shrink by 70% or more over the wire, decompressed transparently by the browser. Whenever the response depends on one of these headers, the server must set Vary so caches key on it, the classic bug being a cache that serves a French page to an English visitor because it ignored Vary: Accept-Language.

The problem each version solves

HTTP/1.1 sends requests one at a time per connection (chapter 3), so a page needing fifty resources is slow. Browsers worked around it by opening about six connections per host, but that's a hack: it's limited, and each connection pays its own handshake. The real fix is to carry many requests on one connection at once, which is multiplexing, and the story of HTTP/2 and HTTP/3 is multiplexing done progressively better.

HTTP/2: many streams, one TCP connection

HTTP/2 keeps HTTP's meaning identical but changes the wire format from text to binary frames, and lets many streams share a single TCP connection, each stream an independent request/response, interleaved frame by frame. One connection now carries all fifty resources concurrently. It adds header compression (HPACK), because repetitive headers (cookies, user-agent) on every request waste bandwidth, and stream prioritisation. The result is a big speed-up over 1.1 for typical pages, and it's why HTTP/2 reached about 80% of top sites.

But HTTP/2 has a flaw it inherited from TCP. All those streams ride one TCP connection, and TCP delivers bytes strictly in order: if one packet is lost, TCP holds back everything that arrived after it until the missing packet is retransmitted, a round trip later. So a single lost packet stalls every stream on the connection, even streams whose own data arrived fine. This is transport-level head-of-line blocking, and on a lossy link (mobile, congested Wi-Fi) it can make HTTP/2 slower than HTTP/1.1's six separate connections, because with six connections a loss only stalls one of them.

HTTP/3: multiplexing on QUIC

HTTP/3 fixes this by abandoning TCP. It runs over QUIC (RFC 9000), a transport built on UDP that provides its own streams, and crucially QUIC's streams are independent: a lost packet blocks only the stream whose data it carried, while every other stream keeps flowing. The head-of-line blocking is gone. QUIC brings three more wins because it was designed from scratch: the transport and TLS handshakes are combined into one round trip (and 0 round trips on resumption), so a fresh HTTPS connection is faster; it supports connection migration, so your download survives switching from Wi-Fi to cellular without starting over; and encryption is built in, not bolted on. The cost is that it's UDP, which some networks throttle or block, so clients fall back to HTTP/2 when QUIC can't get through.

import math

# A simplified model of fetching N resources, to show where each HTTP version wins and loses.
# One "round" is one round-trip time (RTT). Not packet-accurate, but it captures the mechanism.

def http11_rounds(n, connections=6):
    """HTTP/1.1: serial per connection; browsers open ~6 connections per host."""
    return math.ceil(n / connections)                    # each connection does its share serially


def multiplexed(n, rtt_ms, lost_stream=True):
    """HTTP/2 and HTTP/3 both put all N streams in flight at once: ~1 round with no loss.
    The difference is what one lost packet does."""
    base = rtt_ms
    # One packet is lost mid-transfer; recovering it costs ~1 extra RTT.
    h2_completion = base + rtt_ms          # TCP delivers in order, so the gap stalls EVERY stream
    h2_delayed = n
    h3_completion = base                   # QUIC streams are independent...
    h3_one_stream = base + rtt_ms          # ...only the stream on the lost packet waits
    h3_delayed = 1
    return {
        "h2": {"all_streams_done_at_ms": h2_completion, "streams_delayed": h2_delayed},
        "h3": {"most_done_at_ms": h3_completion, "one_stream_at_ms": h3_one_stream,
               "streams_delayed": h3_delayed},
    }


if __name__ == "__main__":
    n, rtt = 20, 50
    print(f"fetching {n} resources, RTT {rtt}ms:\n")
    print(f"HTTP/1.1 (6 connections): {http11_rounds(n)} rounds = {http11_rounds(n) * rtt}ms"
          f"  (serial per connection)")
    print(f"HTTP/2 or HTTP/3 (no loss): 1 round = {rtt}ms  (all {n} streams multiplexed)\n")

    print("now one packet is lost mid-transfer:")
    r = multiplexed(n, rtt)
    print(f"  HTTP/2 (one TCP pipe): all {r['h2']['streams_delayed']} streams wait -> "
          f"{r['h2']['all_streams_done_at_ms']}ms  (TCP head-of-line blocking)")
    print(f"  HTTP/3 (QUIC streams): {n - r['h3']['streams_delayed']} streams done at "
          f"{r['h3']['most_done_at_ms']}ms, only {r['h3']['streams_delayed']} waits to "
          f"{r['h3']['one_stream_at_ms']}ms")

The model shows the whole arc. With no loss, HTTP/2 and HTTP/3 both crush HTTP/1.1 (one round versus four) by multiplexing. Under a single packet loss, HTTP/2's shared TCP pipe stalls all twenty streams while HTTP/3 delays only the one stream that lost a packet. That difference is the entire reason HTTP/3 exists.

What this means for you

Mostly, you turn it on. HTTP/2 and HTTP/3 are enabled at the server, load balancer or CDN, and because the semantics are unchanged, your application code and the headers of chapters 3 and 4 work as before. Two practical consequences, though. Old HTTP/1.1 performance hacks become counter-productive on HTTP/2: concatenating many files into one, "domain sharding" across hostnames, and inlining assets all fight multiplexing, so drop them. And protocol features are attack surface: in 2023 the HTTP/2 Rapid Reset attack (CVE-2023-44487) abused the cheapness of opening and instantly cancelling a stream to generate record-breaking denial-of-service floods, up to 398 million requests a second, a reminder that every new capability is also a new way to be attacked.

What TLS provides

TLS (Transport Layer Security, the successor to SSL) sits between TCP and HTTP and turns a plain, readable connection into a secure one. It provides confidentiality, the bytes are encrypted, so a router, a Wi-Fi snooper or your ISP sees only ciphertext; integrity, any tampering is detected and the connection fails rather than delivering altered data; and authentication, you can be sure the server is who it claims to be. The first two are "just" encryption. The third is the part that makes the web trustworthy and the part most worth understanding, because encrypting a conversation with an impostor buys you nothing.

The handshake

Before any HTTP flows, client and server run a handshake to agree on keys and verify identity. TLS 1.3, the current version, does it in a single round trip:

Two things are happening at once. A key exchange (Diffie-Hellman) lets both sides derive the same secret key without ever sending it, using fresh, ephemeral keys per connection so that stealing the server's long-term key later can't decrypt past traffic (forward secrecy). And authentication: the server sends its certificate chain and signs the handshake, proving it holds the private key for the certificate. Connecting to a real server shows the result:

$ echo | openssl s_client -connect example.com:443 -servername example.com -brief
Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
Peer certificate: CN=example.com
Signature type: ecdsa_secp256r1_sha256
Verification: OK

Certificates and the chain of trust

A certificate binds a public key to a name (example.com), and is signed by a certificate authority (CA). You don't trust the certificate because it says so; you trust it because you can follow its signatures up a chain to a root CA your device already trusts. Your operating system and browser ship with a list of a few hundred trusted root CAs, and validation means: the name matches the site, the dates are current, each certificate was really signed by the next one up, and the top is a trusted root. A real chain looks like this:

subject= CN=example.com                          <- leaf (the site)
issuer = Cloudflare TLS Issuing ECC CA 3
subject= Cloudflare TLS Issuing ECC CA 3         <- intermediate
issuer = SSL.com TLS Transit ECC CA R2
subject= SSL.com TLS ECC Root CA 2022            <- root (in your trust store)

And the validation logic, which every browser runs on every connection, is only this:

from dataclasses import dataclass

# A certificate says "this public key belongs to this name", signed by an issuer. Your browser
# trusts a leaf only if it can follow the signatures up to a root CA it already trusts.
@dataclass
class Cert:
    subject: str        # who this cert is for
    issuer: str         # who signed it (the subject of the cert above)
    signed_by: str      # key id that actually signed it (must match the issuer's key)
    key_id: str         # this cert's own key id
    not_before: int
    not_after: int
    is_ca: bool = False


TRUSTED_ROOT_KEYS = {"root-key-ssl2022"}   # the browser/OS ships a list of these


def verify_chain(chain, hostname, now):
    """chain = [leaf, intermediate(s)..., root]. Returns (ok, reason)."""
    leaf = chain[0]
    if leaf.subject != hostname:
        return False, f"name mismatch: cert is for {leaf.subject}, not {hostname}"
    for i, cert in enumerate(chain):
        if not (cert.not_before <= now <= cert.not_after):
            return False, f"{cert.subject}: outside validity window"
        if i + 1 < len(chain):
            issuer = chain[i + 1]
            if cert.issuer != issuer.subject:
                return False, f"{cert.subject}: issuer name doesn't match next cert"
            if cert.signed_by != issuer.key_id:
                return False, f"{cert.subject}: signature not from the next cert's key"
            if not issuer.is_ca:
                return False, f"{issuer.subject} signed a cert but is not a CA"
    root = chain[-1]
    if root.key_id not in TRUSTED_ROOT_KEYS:
        return False, f"chain ends at {root.subject}, not a trusted root"
    return True, "trusted"


if __name__ == "__main__":
    # each cert's signed_by must equal the key_id of the cert above it (the real signature)
    leaf  = Cert("example.com",  "Issuing CA",   signed_by="k-issuing",         key_id="k-leaf",    not_before=0, not_after=200)
    inter = Cert("Issuing CA",   "Root CA 2022", signed_by="root-key-ssl2022",  key_id="k-issuing", not_before=0, not_after=3650, is_ca=True)
    root  = Cert("Root CA 2022", "Root CA 2022", signed_by="root-key-ssl2022",  key_id="root-key-ssl2022", not_before=0, not_after=7300, is_ca=True)
    chain = [leaf, inter, root]

    print("valid chain:      ", verify_chain(chain, "example.com", now=100))
    print("wrong hostname:   ", verify_chain(chain, "evil.com", now=100))
    print("expired leaf:     ", verify_chain(chain, "example.com", now=250))
    forged = [Cert("example.com", "Issuing CA", signed_by="k-attacker", key_id="k-leaf", not_before=0, not_after=200), inter, root]
    print("forged signature: ", verify_chain(forged, "example.com", now=100))
    # a fully self-consistent chain whose root simply isn't in the trust store
    rogue_root = Cert("Rogue Root", "Rogue Root", signed_by="k-rogue", key_id="k-rogue", not_before=0, not_after=7300, is_ca=True)
    rogue_int  = Cert("Issuing CA", "Rogue Root", signed_by="k-rogue", key_id="k-issuing", not_before=0, not_after=3650, is_ca=True)
    print("untrusted root:   ", verify_chain([leaf, rogue_int, rogue_root], "example.com", now=100))

The checks are exactly the ones an attacker must defeat, and the example shows each failing safely: a certificate for the wrong name, an expired one, one whose signature doesn't come from its stated issuer, and a chain that ends at a root nobody trusts. This is the whole security of HTTPS: anyone can generate a certificate claiming to be your bank, but they can't make it chain to a trusted root without a CA signing it. The weakness, that any of the hundreds of CAs can vouch for any name, is real, and the defences against a misbehaving CA (Certificate Transparency, pinning, shrinking lifetimes) are the subject of the security stage.

SNI, and serving many sites

One server at one IP address can host thousands of HTTPS sites, just as with the Host header (chapter 3), but there's a chicken-and-egg problem: the server must present the right certificate before the encrypted connection exists, so it can't read a Host header yet. The solution is Server Name Indication (SNI): the client puts the hostname in the ClientHello, in the clear, so the server knows which certificate to send. That cleartext hostname is the one piece of a TLS connection an eavesdropper can still see, which is why Encrypted Client Hello (ECH), standardised in 2026, now encrypts the SNI too, closing the last metadata leak.

Mutual TLS, and HTTPS everywhere

Normally only the server proves its identity. With mutual TLS (mTLS) the client presents a certificate as well, so both sides are authenticated, which is how services inside a system authenticate to each other without passwords (the zero-trust model of the production and security stages). And the web has moved to HTTPS by default: browsers mark plain HTTP as insecure, HSTS lets a site insist browsers only ever connect over HTTPS, and certificates are now free (via Let's Encrypt) and short-lived, which means the one rule you must follow is to automate renewal with ACME, because a manually-renewed certificate is an outage waiting for a date.

The problem: HTTP only answers when asked

In the request/response model, the server can't speak until the client asks, so "notify me the moment something happens" doesn't fit. Four approaches bridge the gap, each keeping a connection open longer than the last:

The ladder, rung by rung

• Short polling. The client just asks again every few seconds. Trivial to build, works over any HTTP, but wasteful (most replies are "nothing new") and laggy (updates wait for the next poll). Fine for data that changes slowly and tolerates delay.
• Long polling. The client asks, and the server holds the response open until there's something to send, then answers and the client immediately asks again. Near-real-time with ordinary HTTP, which is why it was the web's workhorse for years, but it's still one message per request and ties up a connection.
• Server-Sent Events (SSE). The client opens one long-lived HTTP response and the server streams events down it as they happen, a simple text format, one per block. It's built into browsers (EventSource), reconnects automatically, and runs over plain HTTP, but it's one-way (server to client) and text-only. Ideal for feeds, notifications, live dashboards, progress, and token streaming from an AI model.
• WebSocket. A single, persistent, full-duplex connection: after an upgrade handshake, either side can send a message at any time, as binary or text, with low overhead. The right tool for chat, multiplayer games, collaborative editing, anything genuinely two-way.

The WebSocket upgrade

A WebSocket is clever about starting: it begins as an ordinary HTTP request that asks to upgrade the connection, so it works through the same ports, proxies and firewalls as HTTPS. The server proves it understood by hashing the client's key with a fixed magic string and returning 101 Switching Protocols; after that, the TCP connection is a WebSocket and no longer speaks HTTP.

import base64
import hashlib

# A WebSocket starts life as an ordinary HTTP request that asks to "upgrade" the connection.
# The server proves it understood by hashing the client's key with a fixed magic string.
WS_MAGIC = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11"   # from RFC 6455


def accept_key(client_key: str) -> str:
    digest = hashlib.sha1((client_key + WS_MAGIC).encode()).digest()
    return base64.b64encode(digest).decode()


def client_upgrade_request(host, path, key):
    return (f"GET {path} HTTP/1.1\r\n"
            f"Host: {host}\r\n"
            f"Upgrade: websocket\r\n"
            f"Connection: Upgrade\r\n"
            f"Sec-WebSocket-Key: {key}\r\n"
            f"Sec-WebSocket-Version: 13\r\n\r\n")


def server_upgrade_response(key):
    return ("HTTP/1.1 101 Switching Protocols\r\n"
            "Upgrade: websocket\r\n"
            "Connection: Upgrade\r\n"
            f"Sec-WebSocket-Accept: {accept_key(key)}\r\n\r\n")


if __name__ == "__main__":
    # The exact example from RFC 6455, section 1.3.
    key = "dGhlIHNhbXBsZSBub25jZQ=="
    print("client key :", key)
    print("accept key :", accept_key(key))
    print("matches RFC:", accept_key(key) == "s3pPLMBiTxaQ9kYGzzhZRbK+xOo=")
    print("\nupgrade handshake over HTTP:")
    print(client_upgrade_request("chat.example", "/ws", key).replace("\r\n", "\\r\\n\n").rstrip())
    print("  --- then ---")
    print(server_upgrade_response(key).replace("\r\n", "\\r\\n\n").rstrip())
    print("\nafter 101, the TCP connection is a bidirectional WebSocket, no longer HTTP.")

The accept-key computation matches the RFC's own test vector exactly, which is how a server (and the browser) confirm the peer really speaks WebSocket and that no cache or proxy mangled the handshake. The magic string serves no cryptographic purpose; it just makes an accidental upgrade impossible.

WebRTC and WebTransport

Two more sit at the specialised ends. WebRTC is built for peer-to-peer real-time media, video calls, voice, between browsers directly, which means solving hard problems TLS and TCP don't: punching through NATs (chapter 14) with STUN and TURN servers, and tolerating loss over UDP because a dropped video frame should be skipped, not retransmitted (chapter 11). It also offers data channels. It's the right, if heavy, tool for conferencing. WebTransport is the newest arrival, reaching cross-browser support in 2026: it runs over HTTP/3 and QUIC, giving multiplexed streams and unreliable datagrams over one connection with no head-of-line blocking (chapter 5), effectively a modern WebSocket that also does the lossy-datagram job WebRTC data channels did. For new client-server real-time work on HTTP/3, it's increasingly the answer; WebSocket remains the safe, universal default.

Calling a function on another machine

Much of a modern system is services calling other services, and RPC (remote procedure call) makes that look like an ordinary function call: you call getUser(42) and get a user back, while a framework handles turning the call into bytes, sending them, and turning the reply back into an object. The convenience is real and the danger is that it hides the network: a local call can't be slow for 200 milliseconds, fail halfway, or vanish, and a remote one can (the fallacies of distributed computing, which the distributed-systems stage covers). So treat RPC as a networked operation with timeouts, retries and error handling, not as a function that happens to live elsewhere.

gRPC and the alternatives

The dominant RPC framework, gRPC, runs over HTTP/2 (chapter 5), sends compact binary messages, and supports streaming in both directions, not just one request and one response but server streams, client streams and bidirectional streams. It generates strongly-typed client and server code from a schema, so a mismatch is a compile error rather than a runtime surprise. That makes it excellent for service-to-service traffic inside a system, where speed and strict contracts matter and humans aren't reading the wire. Its trade-off against plain REST (HTTP plus JSON, chapters 3 and 4) is the usual one: REST is human-readable, browser-native and loosely coupled; gRPC is faster, typed and stricter but not directly usable from a browser (which can't control HTTP/2 frames), needing a gRPC-Web or Connect proxy. Rule of thumb: REST or JSON for public and browser-facing APIs, gRPC for internal service meshes where you control both ends.

Protocol Buffers: the wire format

gRPC serialises with Protocol Buffers (protobuf): you write a schema listing each field with a number, and the encoder puts those numbers, not the field names, on the wire, with the values packed tightly. This is both why protobuf is small and fast and why it evolves gracefully. Here is the whole encoding, small enough to hold in your head:

# Protocol Buffers encode a message by FIELD NUMBER, not field name. That one fact is why
# services that deploy independently can evolve their schemas without breaking each other.
def varint(n):
    out = bytearray()
    while True:
        b = n & 0x7F
        n >>= 7
        out.append(b | (0x80 if n else 0))
        if not n:
            return bytes(out)


def read_varint(data, i):
    result = shift = 0
    while True:
        b = data[i]; i += 1
        result |= (b & 0x7F) << shift
        if not (b & 0x80):
            return result, i
        shift += 7


def encode(fields):
    """fields: {field_number: value}. int -> varint (wire type 0), str -> length-delimited (2)."""
    out = bytearray()
    for num, value in fields.items():
        if isinstance(value, int):
            out += varint((num << 3) | 0) + varint(value)
        else:
            raw = value.encode()
            out += varint((num << 3) | 2) + varint(len(raw)) + raw
    return bytes(out)


def decode(data, schema):
    """schema: {field_number: (name, type)}. Unknown fields are kept but skipped, forward-compat."""
    out, i = {}, 0
    while i < len(data):
        tag, i = read_varint(data, i)
        num, wire = tag >> 3, tag & 0x7
        if wire == 0:
            value, i = read_varint(data, i)
        else:
            length, i = read_varint(data, i)
            value, i = data[i:i + length], i + length
        if num in schema:
            name, typ = schema[num]
            out[name] = value.decode() if typ == "str" else value
        else:
            out[f"<unknown field {num}>"] = value       # old code tolerates new fields
    return out


if __name__ == "__main__":
    wire = encode({1: 42, 2: "widget"})                  # id=42, name="widget"
    print("on the wire:", wire.hex(), f"({len(wire)} bytes)")

    v1 = {1: ("id", "int"), 2: ("name", "str")}
    print("v1 schema decodes:  ", decode(wire, v1))

    # Rename field 2 "name" -> "label": safe, because the wire uses the NUMBER, not the name.
    v2_rename = {1: ("id", "int"), 2: ("label", "str")}
    print("after renaming f2:  ", decode(wire, v2_rename))

    # A new field 3 added by a newer sender; the old decoder keeps it as unknown, no crash.
    newer = encode({1: 42, 2: "widget", 3: 7})
    print("old code, new data: ", decode(newer, v1))

    # DANGER: reusing field number 2 for a new meaning silently corrupts old data.
    v_bad = {1: ("id", "int"), 2: ("price_cents", "int")}
    print("reused number 2:    ", decode(wire, v_bad), "<- price_cents is garbage, not a price")

The message {id: 42, name: "widget"} is ten bytes, and the hex shows why: each field is a tag (the field number and a type) followed by the value, no field names anywhere. JSON would spell out "id" and "name" on every message; protobuf never does.

Schema evolution: the skill that matters

Services deploy independently, so at any moment an old sender may talk to a new receiver or the reverse, and the schema must change without breaking either. Because the wire identifies fields by number, the rules are precise, and the example demonstrates each:

• Renaming a field is safe. The example decodes the same bytes with field 2 called name and then label, both work, because the wire only ever had the number 2. Names are for your code, not the wire.
• Adding a field is safe, as long as it gets a new number. The old decoder, shown reading a message that includes a new field 3, keeps it as an unknown field and carries on, which is forward compatibility: new senders don't break old receivers.
• Never change or reuse a field number. The last line reuses number 2, once a string, now an integer price_cents, and old data silently decodes into garbage: price_cents comes out as the bytes of "widget". No error, just corruption, which is far worse than a crash. When you remove a field, you mark its number reserved so it can never be reused.

These rules, never change or reuse a number, only add with new numbers, are the whole discipline of evolving a binary contract, and they're the same idea as additive, backward-compatible API changes in The Engineer's Shipyard. Get them wrong and you get the quietest, nastiest bug there is: two services that both "work" but disagree about what the bytes mean.

Beyond protobuf

Protobuf is one of a family of schema-based binary formats (Avro and Thrift are others), all solving the same problem: a compact, fast, evolvable encoding with a contract both sides share. The alternative, schemaless JSON, trades size and speed for human-readability and needing no shared schema, which is exactly the right trade for public APIs and anything a person debugs by eye. The choice isn't gRPC-versus-REST as tribes; it's binary-and-typed where you own both ends and want speed, text-and-flexible where you don't.

A reliable stream over an unreliable network

The network layer (IP, chapter 13) is deliberately dumb: it forwards packets and that's all, so a packet can vanish in a full router queue, arrive twice, or overtake an earlier one. TCP's job is to hide all of that and present your program with a clean byte stream, what you write comes out the other end, in order, complete, or the connection fails honestly. It achieves this entirely at the two endpoints (the end-to-end principle, chapter 1), using three mechanisms: a handshake, sequence numbers, and acknowledgement with retransmission.

The three-way handshake

Before any data, the two sides synchronise with three messages: the client sends SYN (with its starting sequence number), the server replies SYN-ACK (acknowledging the client's and sending its own), and the client sends ACK. Now both know each other's sequence numbers and that the path works in both directions.

The handshake costs one round trip before a single byte of data, which is the floor on every new TCP connection's latency, and the reason HTTPS, adding TLS's handshake on top, costs two (chapter 6). A connection is identified by the four-tuple (source IP, source port, destination IP, destination port), so one server can hold millions of connections at once, each distinguished by the client's address and port, and this is also what NAT (chapter 14) rewrites and what QUIC (chapter 5) escapes to survive network changes.

Sequence numbers and acknowledgement

Every byte in the stream has a sequence number, and the receiver sends back cumulative acknowledgements: "I have everything up to byte N, send me N next". If a segment is lost, later segments may still arrive, but the ACK stops advancing, pointing at the gap, and the receiver buffers the out-of-order data until the missing piece arrives. The sender keeps a copy of everything un-acknowledged and retransmits it, on a timeout, or faster when it sees repeated ACKs for the same byte (three duplicate ACKs trigger a fast retransmit). Here is that whole mechanism:

# TCP turns IP's unreliable, out-of-order packets into a reliable, ordered byte stream, using
# sequence numbers, cumulative acknowledgements, and retransmission. Here is that core.
class Receiver:
    def __init__(self):
        self.expected = 0          # next byte we need, in order
        self.buffer = {}           # out-of-order segments, held until the gap fills
        self.delivered = bytearray()

    def on_segment(self, seq, data):
        """Accept a segment; return the cumulative ACK (the next byte we still want)."""
        if seq <= self.expected < seq + len(data):
            self.buffer[seq] = data
        elif seq > self.expected:
            self.buffer[seq] = data                 # arrived early; keep it for later
        while self.expected in self.buffer:          # deliver everything now contiguous
            chunk = self.buffer.pop(self.expected)
            self.delivered += chunk
            self.expected += len(chunk)
        return self.expected                         # cumulative ACK


def send_reliably(message, drop_seqs, log):
    """Send `message` in 4-byte segments over a channel that drops the given seqs ONCE each."""
    rcv = Receiver()
    segments = [(i, message[i:i + 4]) for i in range(0, len(message), 4)]
    acked = 0
    dropped = set(drop_seqs)
    while acked < len(message):
        for seq, data in segments:
            if seq < acked:
                continue                             # already acknowledged, skip
            if seq in dropped:
                dropped.discard(seq)                 # the channel drops it this once
                log.append(f"seq {seq}: LOST")
                continue
            ack = rcv.on_segment(seq, data)
            log.append(f"seq {seq} ({data.decode()!r}) -> ACK {ack}")
            acked = ack
            if acked < seq + len(data):
                break                                # gap: stop, retransmit from here next round
    return bytes(rcv.delivered)


if __name__ == "__main__":
    msg = b"HELLO, NETWORK!"
    log = []
    got = send_reliably(msg, drop_seqs={4}, log=log)   # drop the segment starting at byte 4
    print("\n".join(log))
    print("\ndelivered:", got.decode())
    print("correct despite the loss:", got == msg)

Watch the ACK in the trace. After the segment at byte 4 is lost, the next segment (byte 8) arrives, but the ACK stays at 4, because TCP acknowledges the stream only up to the first gap, that's the cumulative part. The receiver holds byte 8 in its buffer. When the sender retransmits byte 4, the ACK jumps all the way to 12, acknowledging both the retransmitted segment and the buffered one at once. The stream is delivered intact, in order, despite the loss, which is TCP's entire promise kept in a dozen lines.

Teardown, and the states

Closing is a four-message dance: each side sends a FIN and gets an ACK, because the stream is independently closable in each direction. After its final ACK, a side sits in TIME_WAIT for a while before fully releasing the connection, to make sure a late, duplicated packet from this connection can't be mistaken for a new one reusing the same four-tuple. A server with huge numbers of connections in TIME_WAIT is a real operational issue you'll meet, and it's a direct consequence of this safety rule. TCP tracks a connection through a set of states (LISTEN, SYN-SENT, ESTABLISHED, FIN-WAIT, TIME_WAIT and the rest), which the diagnosis tools of chapter 19 let you watch.

Two different "slow down" problems

TCP's reliability (chapter 9) is only half the story; the other half is how fast to send, and there are two independent limits. Flow control stops a fast sender from overwhelming a slow receiver: the receiver advertises, in every ACK, a receive window, how much buffer space it has free, and the sender never has more unacknowledged data in flight than that. It's a simple, explicit signal from the one place that knows. Congestion control is the hard one: it stops senders from overwhelming the network in between, which has no voice to say "stop". The sender must infer congestion, and traditionally the only signal it has is packet loss.

The sawtooth: probe, back off, repeat

Lacking a dial marked "network capacity", TCP discovers it by probing. It starts cautiously and ramps up fast (slow start: double the window each round trip), until it hits a threshold or loses a packet; then it grows gently (congestion avoidance: add one each round trip), nosing upward until loss appears again; and on loss it cuts the window sharply (multiplicative decrease: halve it). The result, additive increase and multiplicative decrease (AIMD), is the characteristic sawtooth:

# TCP can't be told how fast to send, so it probes: ramp up, and back off when loss appears.
# This is the "AIMD sawtooth" of classic congestion control (Reno-style).
def simulate(rounds, ssthresh=16, loss_rounds=()):
    cwnd = 1                     # congestion window, in segments; grows per round-trip
    history = []
    for r in range(rounds):
        phase = "slow-start" if cwnd < ssthresh else "avoidance"
        history.append((r, cwnd, phase))
        if r in loss_rounds:
            ssthresh = max(1, cwnd // 2)     # multiplicative decrease: halve the target
            cwnd = ssthresh                  # (a timeout would reset cwnd to 1 instead)
        elif cwnd < ssthresh:
            cwnd *= 2                         # slow start: double each round (exponential)
        else:
            cwnd += 1                         # congestion avoidance: additive increase (+1/RTT)
    return history


def bdp(bandwidth_mbps, rtt_ms):
    """Bandwidth-delay product: bytes in flight needed to keep the pipe full."""
    return bandwidth_mbps * 1_000_000 / 8 * (rtt_ms / 1000)


if __name__ == "__main__":
    print("round  cwnd  phase          (one '#' per segment)")
    for r, cwnd, phase in simulate(16, ssthresh=16, loss_rounds={6, 12}):
        bar = "#" * cwnd
        print(f"{r:4}  {cwnd:4}  {phase:13}  {bar}")
    print("\nnote the sawtooth: exponential ramp, then +1/round, halve on loss, repeat.")

    need = bdp(100, 80)                        # 100 Mbps link, 80ms RTT
    print(f"\nto fill a 100 Mbps link at 80ms RTT you need {need/1024:.0f} KB in flight.")
    print(f"a default 64 KB window would cap you at {64*1024*8 / (80/1000) / 1e6:.0f} Mbps"
          f" -- far below the link. This is why TCP window scaling exists.")

The code produces exactly this shape: 1, 2, 4, 8, 16 while doubling, then 17, 18 adding one, then a loss halves it to 9, it climbs again, another loss halves it to 7. The average of that sawtooth is the throughput you actually get, which is why a lossy link (chapter 5's mobile case) gives you so much less than its raw bandwidth: every loss knocks the window down.

The bandwidth-delay product

How much data must be "in flight" (sent but not yet acknowledged) to keep a link busy? Exactly the bandwidth-delay product: bandwidth multiplied by round-trip time. The example computes it: to fill a 100 Mbps link with an 80ms RTT, you need about 977 KB in flight at once. If your window is smaller, you stall waiting for ACKs and never reach the link's speed, the old 64 KB maximum window would cap that fast, far link at a miserable 7 Mbps. This is why high-bandwidth, high-latency paths ("long fat networks", a satellite link, a transcontinental fibre) need large windows, and why window scaling was added to TCP. The lesson generalises: on a fast but distant link, throughput is limited by your window and the RTT, not by the bandwidth you're paying for.

CUBIC, BBR, and bufferbloat

Classic loss-based congestion control has a flaw: to detect congestion it must cause loss, which means it keeps network buffers full, and a full buffer adds latency to everything behind it. That's bufferbloat: a big download makes a video call stutter not because bandwidth ran out but because the buffer filled with the download's packets and everything now waits in a long queue. The default algorithm on most systems, CUBIC, is loss-based and prone to this. Google's BBR takes a different approach: instead of waiting for loss, it models the path, measuring the bottleneck bandwidth and the minimum round-trip time, and sends just enough to fill the pipe without filling the buffers, so it keeps throughput high and latency low. As of 2026 CUBIC is still the common default, BBR is widely used by large services, and the fight against bufferbloat also runs in the routers themselves, through smarter queue management. You rarely choose the algorithm, but knowing why a download tanks your latency, and that it needn't, is the point.

The minimal transport

Where TCP is an elaborate machine for reliability and ordering (chapters 9 and 10), UDP is a thin shim over IP. It adds just two things: ports, so the packet reaches the right program, and an optional checksum to detect corruption. There's no connection to set up, no acknowledgements, no retransmission, no ordering, no flow or congestion control. You send a datagram and it either arrives or it doesn't, and you find out neither way. UDP is also message-oriented: each send is one datagram with preserved boundaries, unlike TCP's shapeless byte stream (chapter 9), which is often reason enough to pick it.

This looks like a downgrade, and for fetching a web page it is, which is why the web runs on TCP. But "the network guarantees nothing and neither do I" is precisely what some applications want, because it hands them control over the trade-offs TCP makes for everyone.

When losing a packet is the right call

TCP's cardinal rule, deliver every byte, in order, becomes a liability the moment data has a deadline. Consider a live voice call:

# For real-time media, a packet that arrives late is as useless as one that never arrives.
# This shows why UDP (drop and move on) beats TCP (retransmit and wait) for a live stream.
def realtime_playout(n_frames, frame_ms, lost_frame, rtt_ms):
    """Each audio frame has a playout deadline (frame_ms apart). A frame is lost in transit.
    Compare TCP's in-order recovery with UDP's drop-and-conceal."""
    # TCP: in-order delivery, so the lost frame AND every frame behind it wait for the
    # retransmission (~1 RTT later). All of them miss their real-time deadlines.
    retransmit_arrives = lost_frame * frame_ms + rtt_ms
    tcp_late = [f for f in range(n_frames)
                if f >= lost_frame and f * frame_ms < retransmit_arrives]
    # UDP: the lost frame is skipped (a tiny glitch, concealed); every other frame is on time.
    udp_late = [lost_frame]
    return {"tcp_frames_glitched": len(tcp_late), "udp_frames_glitched": len(udp_late)}


if __name__ == "__main__":
    # 20ms audio frames, one lost at frame 10, 120ms round trip.
    r = realtime_playout(n_frames=30, frame_ms=20, lost_frame=10, rtt_ms=120)
    print("one lost frame in a live call (20ms frames, 120ms RTT):")
    print(f"  TCP (retransmit + wait): {r['tcp_frames_glitched']} frames arrive too late to play")
    print(f"  UDP (drop + conceal):    {r['udp_frames_glitched']} frame glitches, the rest on time")
    print("\nTCP's reliability becomes a liability: it delays good frames to recover a dead one.")
    print("UDP accepts the one loss and keeps the conversation in sync -- the right trade here.")

One audio frame is lost. TCP, insisting on in-order delivery, holds back not just the lost frame but every frame behind it until the retransmission arrives a round trip later, and in a live call that round trip is an eternity: six frames miss their moment to be played, a long audible gap. UDP simply skips the lost frame, the codec conceals the 20-millisecond hole, and the conversation stays in sync: one tiny glitch instead of a stutter. A late packet is a useless packet here, so retransmitting it is worse than forgetting it. That's the whole case for UDP.

Where UDP belongs

• Real-time media (voice, video, live streaming): skip losses and conceal them; never stall the stream to recover old data. This is why WebRTC (chapter 7) uses UDP.
• Live games: send the latest world state; a dropped older state is worthless, so don't retransmit it, send the new one.
• DNS (chapter 15): one tiny request, one tiny reply; a connection handshake would cost more than the query, so DNS uses UDP and retries at the application layer if an answer doesn't come.
• A foundation to build on: when you want some of TCP's guarantees but not all, or want control TCP can't give, you build your own transport over UDP. That's exactly what QUIC (chapter 5) is: reliability and ordering per stream, its own congestion control, and encryption, all on top of UDP, in user space, free of TCP's rigidity.

What you take on, and the dangers

Choosing UDP means choosing responsibility. Anything you need, reliability, ordering, rate limiting, you must add yourself or deliberately do without. Two dangers in particular. First, no congestion control means a naive UDP application can flood a network and contribute to congestion collapse, which is why every serious UDP protocol (including QUIC) implements congestion control of its own, and why you should never just blast datagrams as fast as you can. Second, amplification attacks: because UDP has no handshake, an attacker can spoof a victim's address as the source of a small request to a server that replies with a much larger response (DNS and NTP are classic amplifiers), turning the server into an unwitting weapon aimed at the victim. It's a reason UDP services must limit response sizes to unverified senders, and a reason the Mirai-era DDoS attacks hit so hard.

The API every program shares

Underneath every HTTP library and web framework is the Berkeley socket API, a small set of system calls, unchanged for decades, that give a program a file-like handle onto a TCP or UDP connection. A server creates a socket, binds it to an address and port, listens, and accepts connections, getting a fresh socket for each one; a client creates a socket and connects to an address; both then send and recv bytes and close when done. That's the whole vocabulary, and this stage's deliverable, an HTTP server on raw sockets, is exactly these calls plus the HTTP parsing of chapter 3. Here is a real, working pair:

import socket
import struct
import threading

# A socket is the OS handle for a network connection. Every client and server you have used,
# browsers, curl, databases, is built on these calls. Here is a real, working pair.
def recv_exact(conn, n):
    """TCP is a byte STREAM, not messages: recv() may return fewer bytes than asked.
    To read a whole message you must loop. This is the single most common socket bug."""
    buf = b""
    while len(buf) < n:
        chunk = conn.recv(n - len(buf))
        if not chunk:
            raise ConnectionError("peer closed mid-message")
        buf += chunk
    return buf


def recv_message(conn):
    """Frame messages with a 4-byte length prefix, since TCP won't do it for us (chapter 9)."""
    (length,) = struct.unpack(">I", recv_exact(conn, 4))
    return recv_exact(conn, length)


def send_message(conn, data):
    conn.sendall(struct.pack(">I", len(data)) + data)


def server(ready, host="127.0.0.1"):
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)         # a TCP socket
    s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
    s.bind((host, 0))                                             # 0 = pick a free port
    s.listen()                                                    # become a listening server
    ready.append(s.getsockname())                                # tell the client our port
    conn, peer = s.accept()                                      # blocks until a client connects
    with conn:
        msg = recv_message(conn)
        send_message(conn, b"echo: " + msg)                      # echo it back, framed
    s.close()


if __name__ == "__main__":
    ready = []
    t = threading.Thread(target=server, args=(ready,))
    t.start()
    while not ready:
        pass
    host, port = ready[0]
    print(f"server listening on {host}:{port}")

    client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    client.connect((host, port))                                 # the TCP handshake happens here
    print("connected; local 4-tuple:", client.getsockname(), "->", client.getpeername())
    send_message(client, b"hello sockets")
    reply = recv_message(client)
    client.close()
    t.join()
    print("server replied:", reply.decode())
    print("round trip over a real socket:", reply == b"echo: hello sockets")

It genuinely runs: the server binds to a free port, the client connects (the TCP handshake of chapter 9 happens inside connect), sends a message, and gets it echoed back. Notice the four-tuple printed, (127.0.0.1, 49172) -> (127.0.0.1, 49171), the source and destination addresses and ports that uniquely identify this connection, which is why one listening server can hold thousands of simultaneous connections on a single port: each is a different four-tuple, and each gets its own socket from accept.

The gotcha: it's a stream, not messages

The single most common socket bug follows directly from chapter 9: TCP is a byte stream with no message boundaries. Your one send of 1000 bytes may arrive as one recv of 1000, or two of 500, or 1000 of one, and two sends may arrive merged into a single recv. The network decides, based on segmentation and timing, not on how you called send. So you cannot assume recv returns a whole message; you must frame the data yourself, and loop until you have all of it. The example does this with a 4-byte length prefix and a recv_exact loop, and HTTP (chapter 3) does it with Content-Length or chunked encoding. Code that does data = conn.recv(4096) and parses data as a complete message works on your laptop and fails in production the first time a message is split across packets, which is a rite of passage every socket programmer goes through once.

Handling many connections

The example serves one connection and exits. A real server must handle many at once, which is where the deliverable's "concurrent connections" requirement bites, and there are three classic approaches:

• A thread (or process) per connection. Simple to write: accept in a loop, hand each new socket to a worker. It's fine for hundreds of connections but each thread costs memory and the OS struggles past some thousands, the historical C10k problem (ten thousand connections).
• An event loop (async I/O). One thread watches many sockets at once with select, epoll or kqueue, doing a little work on whichever is ready, never blocking on any one. This is how nginx, Node.js and Python's asyncio handle tens of thousands of connections cheaply, and it's why the networking world went event-driven.
• A hybrid: a pool of threads, each running an event loop, to use every CPU core, which is what most high-performance servers actually do.

The key realisation is that a networked server spends most of its time waiting, for the next byte, for the client to read, so the scarce resource isn't CPU but the ability to wait on many things at once without a thread tied up per wait. That's the whole motivation for non-blocking I/O.

UDP sockets, and the shape of the two

A UDP socket (chapter 11) uses the same API with a different type (SOCK_DGRAM instead of SOCK_STREAM) and a simpler shape: no listen, no accept, no connection at all, you just sendto an address and recvfrom whoever sends you a datagram. Each recvfrom returns exactly one datagram with its boundary intact, the message-oriented nature from chapter 11, so the stream-framing gotcha doesn't apply, but loss and reordering do. The two socket types are the API's reflection of the two transports: a stream you must frame, or datagrams you must make reliable.

Addresses, and the two versions

IP's one job is to give every host an address and deliver packets to it. There are two versions in use. IPv4 uses 32-bit addresses, written as four bytes (192.168.1.50), which gives about 4.3 billion, long since exhausted, which is why we have NAT (chapter 14) and IPv6. IPv6 uses 128-bit addresses, written in hex groups (2001:db8:abcd:12::1), a space so vast (3.4 × 10³⁸) that every device can have a real, globally routable address with no sharing. IPv6 passed 50% of Google's traffic in 2026 and keeps climbing, so you need to be fluent in both; the concepts are the same, only the size differs.

CIDR: the network/host split

An address alone doesn't say where the network ends and the host begins; a prefix length does. 192.168.1.0/24 means "the first 24 bits are the network, the last 8 identify hosts within it", which is CIDR notation, and it's the language of every routing table and firewall rule. The arithmetic is worth seeing concretely:

import ipaddress

# An IP address plus a prefix length (CIDR) splits into a network part and a host part.
# The /N means "the first N bits identify the network"; the rest identify the host.
def describe(cidr):
    net = ipaddress.ip_network(cidr, strict=False)
    hosts = list(net.hosts())
    return {
        "network": str(net.network_address),
        "prefix": net.prefixlen,
        "total_addresses": net.num_addresses,
        "usable_hosts": len(hosts),
        "first_host": str(hosts[0]) if hosts else None,
        "last_host": str(hosts[-1]) if hosts else None,
        "broadcast": str(net.broadcast_address) if net.version == 4 else "(none in IPv6)",
    }


def in_network(ip, cidr):
    return ipaddress.ip_address(ip) in ipaddress.ip_network(cidr)


if __name__ == "__main__":
    for k, v in describe("192.168.1.0/24").items():
        print(f"  {k:16}: {v}")

    print("\nsplitting a /24 into four /26 subnets (e.g. one per team):")
    for sub in ipaddress.ip_network("192.168.1.0/24").subnets(new_prefix=26):
        print(f"  {sub}  ({sub.num_addresses - 2} usable hosts)")

    print("\nrouting is a membership test (longest-prefix match):")
    for ip in ["192.168.1.50", "192.168.2.50", "10.0.0.1"]:
        print(f"  {ip:14} in 192.168.1.0/24? {in_network(ip, '192.168.1.0/24')}")

    print("\nspecial ranges you must recognise:")
    for ip in ["10.0.0.5", "127.0.0.1", "169.254.169.254", "8.8.8.8"]:
        a = ipaddress.ip_address(ip)
        kind = ("private" if a.is_private and not a.is_loopback and not a.is_link_local
                else "loopback" if a.is_loopback else "link-local" if a.is_link_local
                else "global")
        print(f"  {ip:16} {kind}")

    v6 = ipaddress.ip_network("2001:db8:abcd:12::/64")
    print(f"\nIPv6 {v6}: {v6.num_addresses:.3g} addresses in one standard /64 subnet")

A /24 holds 256 addresses, of which 254 are usable hosts (the first is the network address, the last is the broadcast address in IPv4). Shrinking the prefix grows the network; growing it shrinks it, a /26 has 64 addresses, a /16 has 65,536. Subnetting is splitting one network into smaller ones, as the example splits a /24 into four /26s, which is how an organisation carves its address space into per-team, per-service or per-availability-zone pieces.

Routing is a membership test

Here's why the split matters: a router doesn't store a route for every one of the billions of hosts; it stores routes for networks, and to forward a packet it finds which network the destination address belongs to. The example's in_network check is that decision in miniature. Real routers do longest-prefix match: a packet for 192.168.1.50 might match both 192.168.0.0/16 and 192.168.1.0/24, and the router picks the more specific one (the longer prefix), because it's the more precise route. This aggregation, millions of hosts summarised as a handful of network prefixes, is what keeps the internet's routing tables merely enormous rather than impossible, and it's the mechanism chapter 14 builds on.

Special addresses to recognise

Not every address is a public destination, and confusing the categories causes real bugs and real breaches. The ranges to know: private addresses (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) are reusable inside any network and never routed on the public internet, which is what NAT exists to bridge; loopback (127.0.0.1) is the host talking to itself; and link-local (169.254.0.0/16) is for one network segment only, including the cloud metadata address 169.254.169.254 whose role in the Capital One breach the security stage covers. The example classifies each, and that classification is exactly what a server-side-request-forgery guard checks before fetching a URL (The Engineer's Citadel, chapter 6).

Why IPv6, and why it took so long

IPv4 ran out of addresses more than a decade ago; IPv6 is the designed-in fix, with room for every device to have its own global address and no need for the address-sharing contortions of NAT. Its slow adoption, 18 years to reach half of traffic, is a case study in network effects: there was no single day to switch, every participant had to move independently, and NAT worked well enough to remove the urgency. The practical upshot for you: write code and infrastructure that handles both, parse and store addresses with a library that understands IPv6 (as the example does), never assume an address is four dotted numbers, and know that on an IPv6 network there's often no NAT, so every host is directly addressable, which changes how you think about firewalls.

Routing: local decisions, global delivery

No router knows the whole path to a destination; each one just knows the next hop. A packet for a distant server is handed from your machine to your gateway, to your ISP, across the internet's backbone, router by router, each consulting its routing table, matching the destination against its known networks by longest prefix (chapter 13), and forwarding one hop closer. The destination is reached not by anyone planning the route but by a chain of local decisions, which is what makes the internet both scalable (no central map) and resilient (a packet routes around a failed link). Your own machine's table is tiny, usually just "my local network is here, everything else goes to the default gateway", and the complexity lives in the core.

Autonomous systems and BGP

The internet is divided into about a hundred thousand autonomous systems (AS), independently operated networks, ISPs, cloud providers, large companies, each with a number. Between them, routes are exchanged by the Border Gateway Protocol (BGP): each AS announces "I can reach these network prefixes", and these announcements propagate so every AS learns a path to every prefix. BGP is the internet's control plane, and its defining feature is that it runs on trust: historically an AS could announce a prefix it didn't own and others would believe it, which is how BGP hijacks (accidental or malicious) have rerouted traffic for whole countries. It's also how the internet's most dramatic outages happen: in 2021 a configuration error made Facebook withdraw the BGP routes to its own DNS servers, and in minutes Facebook, Instagram and WhatsApp vanished from the internet entirely, not because the servers were down but because no one could find a route to them. Routing is the layer that decides whether you exist on the internet at all, and it's secured by convention more than by cryptography (though RPKI is slowly changing that).

NAT: sharing a scarce address

IPv4 ran out of addresses (chapter 13), so almost no home or office device has a public one. Instead they sit on private addresses behind a router doing Network Address Translation: on the way out, the router rewrites the packet's private source address and port to its own single public address and a chosen port, remembering the mapping; on the way back, it reverses the rewrite and delivers the reply to the right inside device. One public address serves a whole household or building. Here is the mechanism:

# NAT lets many devices on private addresses share one public IP. The NAT box rewrites the
# source address and port on the way out, and reverses the rewrite on the replies coming back.
class NAT:
    def __init__(self, public_ip):
        self.public_ip = public_ip
        self.next_port = 40000
        self.outbound = {}   # (inside_ip, inside_port, dst) -> public_port
        self.inbound = {}    # public_port -> (inside_ip, inside_port)

    def translate_outbound(self, inside_ip, inside_port, dst):
        key = (inside_ip, inside_port, dst)
        if key not in self.outbound:
            port = self.next_port
            self.next_port += 1
            self.outbound[key] = port
            self.inbound[port] = (inside_ip, inside_port)
        port = self.outbound[key]
        return (self.public_ip, port)          # what the internet sees as the source

    def translate_inbound(self, public_port):
        """A reply arrives for public_ip:public_port; map it back to the real inside host."""
        return self.inbound.get(public_port)   # None = unsolicited; NAT drops it


if __name__ == "__main__":
    nat = NAT("203.0.113.7")                    # the home router's single public IP
    dst = ("93.184.216.34", 443)               # some web server

    # Two different devices behind the router open connections to the same server.
    laptop = nat.translate_outbound("192.168.1.10", 51514, dst)
    phone  = nat.translate_outbound("192.168.1.23", 51514, dst)   # same inside port, by chance
    print("laptop  192.168.1.10:51514 ->", f"{laptop[0]}:{laptop[1]}")
    print("phone   192.168.1.23:51514 ->", f"{phone[0]}:{phone[1]}")
    print("both appear to the internet as one IP:", laptop[0] == phone[0] == "203.0.113.7")

    # Replies come back addressed to the public IP + the mapped port; NAT sends each home.
    print("\nreply to port", laptop[1], "-> inside", nat.translate_inbound(laptop[1]))
    print("reply to port", phone[1],  "-> inside", nat.translate_inbound(phone[1]))
    print("unsolicited packet to port 55555 ->", nat.translate_inbound(55555), "(dropped)")

What NAT breaks

NAT quietly solved address exhaustion and quietly broke the internet's original promise that any host can talk to any host. Because the mapping is created by outbound traffic, an inbound connection to a device behind NAT has nowhere to go, the router has no table entry for it, so it's dropped (the example's last line). That's why running a server at home needs port forwarding, why peer-to-peer apps like video calls need hole-punching through STUN and relaying through TURN servers (the machinery behind WebRTC, chapter 7), and why "just connect directly to the other user" is so much harder than it sounds. At the ISP scale, carrier-grade NAT (CGNAT) shares one public address among many customers, compounding the problem and meaning a source IP no longer identifies even a household. IPv6, with an address for every device, removes the need for NAT entirely, which is one of the strongest arguments for finishing the migration.

Names to addresses

Packets travel to IP addresses (chapter 13), but nobody types 93.184.216.34; they type example.com. The Domain Name System is the distributed directory that translates between them, and it runs before nearly every connection, so its speed and reliability underpin everything above it. It's also a system you'll configure often, pointing a domain at a server, a CDN, a mail provider, so understanding its hierarchy and its caching is daily-useful, not just theory.

The hierarchy, and who does the walking

No single server knows every name; the namespace is a tree, resolved from the top. A name like example.com is found by asking the root servers "who handles .com?", then the .com servers "who handles example.com?", then example.com's own authoritative servers "what's the address?". Your device doesn't do this walk itself; it asks a recursive resolver (your ISP's, or a public one like Cloudflare's 1.1.1.1 or Google's 8.8.8.8), which does the walking and hands back the answer.

# DNS resolves a name by walking a hierarchy: root -> top-level domain -> authoritative server.
# A recursive resolver does the walking and caches answers for their TTL, so most lookups are free.
ROOT = {".com": "tld-com-server", ".org": "tld-org-server"}
TLD = {"tld-com-server": {"example.com": "ns.example.com", "shop.com": "ns.shop.com"}}
AUTH = {"ns.example.com": {"example.com": ("93.184.216.34", 3600)}}   # (IP, TTL seconds)


class Resolver:
    def __init__(self):
        self.cache = {}        # name -> (ip, expires_at)
        self.queries = 0       # network queries actually made

    def resolve(self, name, now):
        cached = self.cache.get(name)
        if cached and now < cached[1]:
            return cached[0], "CACHE (0 queries)"
        # Walk the hierarchy from the root, counting each network hop.
        tld = "." + name.split(".")[-1]
        tld_server = ROOT[tld];                 self.queries += 1    # ask the root
        auth_server = TLD[tld_server][name];    self.queries += 1    # ask the TLD
        ip, ttl = AUTH[auth_server][name];      self.queries += 1    # ask the authoritative server
        self.cache[name] = (ip, now + ttl)
        return ip, "RESOLVED (root -> TLD -> authoritative, 3 queries)"


if __name__ == "__main__":
    r = Resolver()
    for now in (0, 100, 5000):            # TTL is 3600s
        ip, how = r.resolve("example.com", now)
        print(f"t={now:5}s  example.com -> {ip}  [{how}]")
    print(f"\ntotal network queries for 3 lookups: {r.queries}")
    print("first lookup walks the hierarchy; within the TTL it's cached; after 3600s it re-resolves.")
    print("TTL is the trade-off: long = fast & cacheable but slow to change; short = agile but chatty.")

The first lookup walks the hierarchy (three queries); the second, within the TTL, is served from the resolver's cache with no network at all; after the TTL expires, it re-resolves. Caching happens at every level, the resolver, your OS, your browser, which is why DNS feels instant and why most queries never reach the authoritative servers.

Records, and the TTL trade-off

A name can carry many kinds of record: A (IPv4 address), AAAA (IPv6), CNAME (an alias to another name, how you point a domain at a CDN), MX (mail servers), TXT (arbitrary text, used for domain verification and email anti-spoofing), and NS (the authoritative nameservers). A real query shows a domain often returns several A records, which is the simplest load balancing there is: clients spread across the addresses. Every record has a TTL, and it's a genuine trade-off: a long TTL (a day) means fast, cacheable, resilient lookups but slow changes, if you need to move a service, the old address lingers in caches worldwide until the TTL expires; a short TTL (a minute) makes changes and failover quick but multiplies query load and dependence on your DNS being up. Teams routinely lower the TTL days before a planned migration so the cut-over is fast, then raise it again.

Why it's always DNS

DNS sits under everything, so when it breaks, healthy services become unreachable and the cause is rarely obvious, hence the operations proverb. The cautionary record is long: in 2016 a DDoS attack on the DNS provider Dyn took Twitter, Reddit, Netflix and others offline without touching any of them, because their names couldn't be resolved. In 2021 Facebook withdrew the routes to its own DNS servers and vanished for six hours (chapter 14). In 2025 a race condition in AWS's automated DNS management briefly emptied the address record for a core DynamoDB endpoint in its busiest region, and the gap cascaded into a 15-hour outage across much of the internet. The pattern repeats: DNS is a critical dependency with no blast-radius limit, its failures look like everyone else's outage, and its automation and caching create subtle, delayed, hard-to-diagnose faults. When something is unreachable, check DNS first, you'll be right more often than you'd like.

Privacy, and the modern layer

DNS was historically sent in plaintext, so your ISP, or anyone on the path, could see every domain you looked up, a map of your browsing even over HTTPS. The fix is encrypted DNS: DoH (DNS over HTTPS, on port 443, blending in with web traffic) and DoT (DNS over TLS, on its own port), now a meaningful and growing share of queries and the default in some browsers. There's also DNSSEC, which cryptographically signs records so a resolver can detect a forged answer (addressing integrity, not privacy), though its adoption remains small. For a builder, the practical points are: prefer a resolver that supports encrypted DNS, remember that the SNI (chapter 6) still leaked the hostname until ECH, and know that DNS is now one more encrypted, authenticated layer rather than the open postcard it used to be.

Two numbers, not one

Bandwidth is how much data per second a link can carry; latency is how long one trip takes. They're independent: a satellite link can have huge bandwidth and terrible latency, and a short cable the reverse. People say "faster internet" meaning bandwidth, but for most of what you build, loading a page, an API call, a database query, the experience is dominated by latency, because the work is a conversation of small messages, each waiting a round trip for its answer. Stuart Cheshire's old essay title says it: "It's the latency, stupid."

Latency has a floor, and it's physics

Light in fibre travels about 200,000 km per second, two-thirds of its speed in vacuum, which is roughly 200 km per millisecond. That sets a hard floor no engineering can cross:

# Latency and bandwidth are different resources. Latency is set by distance and the speed of
# light; you cannot optimize code past it. The only way to cut it is fewer, shorter round trips.
FIBER_KM_PER_MS = 200          # light in fibre travels ~200 km per millisecond (about 2/3 of c)


def rtt_ms(distance_km, route_factor=1.5):
    """Round-trip time floor for a distance. route_factor: real fibre isn't a straight line."""
    one_way = distance_km / FIBER_KM_PER_MS * route_factor
    return round(2 * one_way)


def cold_https_ms(rtt):
    """A first visit: DNS + TCP + TLS + the HTTP request, each about one round trip."""
    return 4 * rtt


def transfer_ms(size_kb, rtt, bandwidth_mbps):
    """Total time = latency (round trips) + the bytes / bandwidth."""
    bits = size_kb * 1024 * 8
    download_ms = bits / (bandwidth_mbps * 1_000_000) * 1000
    return cold_https_ms(rtt) + download_ms


if __name__ == "__main__":
    print("round-trip floor by distance (fibre):")
    for place, km in [("same city", 50), ("New York-London", 5500), ("New York-Sydney", 16000)]:
        print(f"  {place:18} {km:6} km -> ~{rtt_ms(km):3} ms RTT")

    print("\na cold HTTPS page, far origin vs nearby CDN:")
    far, near = rtt_ms(16000), rtt_ms(50)
    print(f"  distant origin (RTT {far}ms): {cold_https_ms(far)} ms before the HTML even starts")
    print(f"  nearby CDN     (RTT {near}ms): {cold_https_ms(near)} ms  ({cold_https_ms(far)//cold_https_ms(near)}x faster, same code)")

    print("\nlatency vs bandwidth: when does each dominate? (RTT 150ms, 100 Mbps)")
    for size, label in [(10, "small (10 KB, typical asset)"), (10_000, "large (10 MB, video)")]:
        total = transfer_ms(size, 150, 100)
        lat = cold_https_ms(150)
        print(f"  {label:30} {total:7.0f} ms total, {lat/total*100:3.0f}% is latency")
    print("\nfor small objects (most of the web), latency dominates -- more bandwidth won't help.")

New York to Sydney is about 16,000 km, so a round trip can't beat ~160ms in a straight line and is ~240ms over real fibre routes, per round trip, forever, no matter how fast the servers or your code. This is the single most important fact at this layer: you cannot optimise your way under the speed of light. You can only make fewer round trips, or make them shorter.

The latency budget: count the round trips

A "slow page" is usually a stack of round trips, each paying that distance tax. A cold HTTPS request is about four round trips before the HTML even begins, DNS, TCP handshake, TLS handshake, then the request itself (chapters 6 and 9), and the example shows what that costs at distance:

This is why every latency win in this book, connection reuse and keep-alive (chapter 3), HTTP/2 and HTTP/3 multiplexing (chapter 5), TLS resumption and QUIC's 0-RTT (chapters 6, 5), and CDNs (chapter 17), is about the same two moves: remove round trips, or shorten them. A "chatty" API that makes ten sequential calls to render a page pays ten times the round trip, which across a continent is the difference between snappy and sluggish, and no faster server fixes it; batching the ten into one does.

When bandwidth matters, and when it doesn't

Bandwidth takes over only for bulk transfer. The example's last block makes it concrete: fetching a 10 KB asset at 150ms RTT is essentially all latency (the bytes take under a millisecond to transfer, the round trips take hundreds), so doubling the user's bandwidth changes nothing; fetching a 10 MB video is bandwidth-bound, and there more bandwidth helps directly. Most of the web is small objects, which is why most web performance work is latency work, and why "we upgraded to a gigabit connection and the site is still slow" is such a common, puzzled complaint, the site was never bandwidth-bound. Know which regime you're in before you optimise. And remember the bandwidth-delay product from chapter 10: on a fast, distant link, even bulk transfer is throttled by your window and the RTT, so latency sneaks back in.

Jitter and the tail

Two refinements matter in production. Jitter is the variation in latency, round trips aren't all equal, because packets queue behind others (chapter 10's bufferbloat), and jitter is what makes a video call choppy even when average latency is fine. And the number that matters to users isn't the average but the tail: the 95th or 99th percentile, the slowest requests, because a user who makes twenty requests to render one page experiences the slowest of the twenty, not the median. Measure and optimise the tail (p95, p99), not the mean, a lesson the reliability and performance stages return to.

Why a CDN exists

One origin server, however powerful, is far from most of your users, and chapter 16 showed distance is a latency tax no code can dodge. A content delivery network solves this by distance: it runs servers, points of presence (PoPs), in hundreds of cities, caches your content on them, and serves each user from the nearest one. The round trip that was 240ms to a distant origin becomes a few milliseconds to a local PoP. A CDN does four jobs at once: it shortens the distance for latency; it caches static content at the edge (chapter 4's caching, now global) so the origin is barely touched; it absorbs traffic spikes and DDoS because its edge is vastly larger than your origin; and it terminates TLS near the user, so even the handshake round trips are short. Increasingly it also runs your code at the edge.

Anycast: one address, many places

How does a user reach the nearest PoP without the application choosing one? The elegant answer is anycast: the CDN announces the same IP address from every PoP via BGP (chapter 14), and the internet's own routing delivers each user's packets to whichever PoP is nearest in network terms. No DNS trickery, no client logic, routing does it.

# A CDN announces ONE IP address from many points of presence (PoPs) at once, via BGP.
# The internet's own routing then delivers each user to the nearest PoP. No DNS trickery needed.
POPS = {                       # network "distance" (ms) from each region to each PoP
    "london":   {"eu-user": 8,  "us-user": 75, "asia-user": 160},
    "virginia": {"eu-user": 80, "us-user": 10, "asia-user": 180},
    "singapore":{"eu-user": 170,"us-user": 185,"asia-user": 12},
}


def route(user, live_pops):
    """Anycast sends the user to the PoP that's nearest by network distance (BGP picks it)."""
    return min(live_pops, key=lambda pop: POPS[pop][user])


if __name__ == "__main__":
    users = ["eu-user", "us-user", "asia-user"]
    live = list(POPS)
    print("normal operation (all PoPs up):")
    for u in users:
        pop = route(u, live)
        print(f"  {u:10} -> {pop:10} ({POPS[pop][u]} ms)")

    print("\nthe Singapore PoP fails; BGP withdraws its route, traffic reroutes automatically:")
    live_after = [p for p in live if p != "singapore"]
    for u in users:
        pop = route(u, live_after)
        print(f"  {u:10} -> {pop:10} ({POPS[pop][u]} ms)")
    print("  the Asia user now takes a slower but working path -- no config change, just routing.")

    # Edge caching offloads the origin: only misses travel the long way back.
    hit_rate = 0.95
    print(f"\nwith a {hit_rate:.0%} edge cache hit rate, the origin sees only "
          f"{(1-hit_rate)*100:.0f}% of requests -- a 20x reduction in origin load.")

The example shows both halves: in normal operation each user reaches their nearest PoP (single-digit milliseconds); when the Singapore PoP fails, its BGP route is withdrawn and the Asia user is rerouted to London, slower, but working, with no configuration change and no human action. That automatic rerouting is anycast's quiet superpower, and the edge caching above it means the origin, even in normal times, sees only the small fraction of requests that miss the cache (a 95% hit rate is a twenty-fold reduction in origin load).

The cache hierarchy and the edge

CDNs layer caches: a request hits the nearest edge PoP; on a miss it may check a regional cache, and only then the origin (often through a single "origin shield" PoP, so a cold cache doesn't stampede the origin, the thundering-herd problem the reliability stage covers). Static content (images, scripts, videos) caches beautifully; dynamic, per-user content doesn't, which is why the private/no-store directives of chapter 4 matter so much at the edge, a mis-marked personalised response cached at a PoP is served to strangers. The frontier is edge compute: running your code at the PoPs (Cloudflare Workers and the like), so even dynamic logic executes near the user rather than a continent away, blurring the line between "CDN" and "where your app runs".

The shared-dependency risk

A CDN concentrates enormous value and enormous risk. Because the same configuration and code run at every PoP, a bad change propagates everywhere in seconds, exactly as fast as the content it's meant to speed up. In 2021 a single customer's valid configuration change triggered a latent bug at Fastly and 85% of its network returned errors within a minute, taking down a swathe of the internet, before being fixed in under an hour. In 2025 an oversized internal configuration file crashed Cloudflare's proxies across its global edge, degrading much of the web for hours. The lessons are the ones that recur in this volume: a CDN is a single point of failure for everything behind it, so its own changes need staging and rollback like any deploy; anycast spreads a fix globally but spreads a bug just as globally; and depending on one provider for your entire presence is a risk worth weighing against the very real benefits. The benefits usually win, but go in with eyes open.

Two places to balance

Everything in this volume has built toward a request arriving at a server. But "a server" is a lie: behind one address is a pool of identical servers, and a load balancer (LB) decides which one handles each request. It does two jobs that matter equally: it spreads load so no server is overwhelmed, and it provides availability by routing around servers that fail, so one dead instance is invisible to users. Where the LB operates in the layered model (chapter 1) decides what it can see and do.

A layer-4 load balancer works at the transport layer. It sees TCP connections, the 4-tuple from chapter 9, and nothing above: it can't read the URL or headers, because those live inside the byte stream (and under TLS, they're encrypted). So it picks a backend per connection, typically by hashing the 4-tuple so a given client sticks to one backend, and shovels bytes through. It's simple, extremely fast, and protocol-agnostic, it balances anything over TCP, not just HTTP.

A layer-7 load balancer terminates the connection, reads the HTTP request, and routes per request on what it can now see: path, Host, headers, cookies. It can send /api/* to one pool and /static/* to another, retry a failed request on a different backend, add tracing headers, enforce rate limits, and do TLS termination. It's doing far more work, so it's slower per request than L4, but it's where almost all modern application routing happens.

import zlib

# ---------------------------------------------------------------------------
# A layer-4 load balancer works at the transport layer. It sees a TCP
# connection -- the 4-tuple (src ip, src port, dst ip, dst port) -- and nothing
# above it. It can't read the URL or headers (that's inside the encrypted,
# or at least higher-layer, byte stream). So it picks a backend per CONNECTION
# and sends the whole connection there, usually by hashing the 4-tuple so the
# same client sticks to the same backend.
def l4_route(four_tuple, backends):
    h = zlib.crc32(four_tuple.encode())
    return backends[h % len(backends)]

backends = ["app-a", "app-b", "app-c"]
conns = ["10.0.0.5:51000->lb:443", "10.0.0.5:51001->lb:443", "10.0.0.9:62000->lb:443"]
print("layer 4 (routes whole connections by 4-tuple hash):")
for c in conns:
    print(f"  {c:32} -> {l4_route(c, backends)}")

# ---------------------------------------------------------------------------
# A layer-7 load balancer terminates the connection, reads the HTTP request,
# and routes PER REQUEST on what it can see: the path, the Host, headers,
# cookies. It can send /api to one pool and /img to another, retry a failed
# request on another backend, and add/inspect headers.
POOLS = {"api": ["api-1", "api-2"], "static": ["cdn-1", "cdn-2"]}

def l7_route(method, path, healthy):
    pool = "api" if path.startswith("/api/") else "static"
    live = [b for b in POOLS[pool] if healthy.get(b, True)]
    if not live:
        return pool, None            # no healthy backend -> 503
    # least-connections would go here; round-robin by a tiny counter instead
    idx = l7_route.rr.setdefault(pool, 0)
    l7_route.rr[pool] = (idx + 1) % len(live)
    return pool, live[idx]
l7_route.rr = {}

print("\nlayer 7 (routes each request by path, across healthy backends):")
healthy = {"api-1": True, "api-2": True, "cdn-1": True, "cdn-2": True}
reqs = [("GET", "/api/orders"), ("GET", "/api/users"), ("GET", "/img/logo.png"), ("GET", "/img/hero.jpg")]
for m, p in reqs:
    pool, b = l7_route(m, p, healthy)
    print(f"  {m} {p:16} -> {pool} pool -> {b}")

# ---------------------------------------------------------------------------
# Health checks: the LB probes each backend and stops sending to one that
# fails. This is the whole point -- a dead backend is routed around in seconds,
# with no request landing on it, no user noticing.
print("\napi-2 fails its health check; the LB routes around it:")
healthy["api-2"] = False
l7_route.rr = {}
for m, p in [("GET", "/api/a"), ("GET", "/api/b"), ("GET", "/api/c")]:
    pool, b = l7_route(m, p, healthy)
    print(f"  {m} {p:10} -> {b}")
assert all(l7_route(m, p, healthy)[1] == "api-1" for m, p in [("GET","/api/x")])
print("  every /api request now goes to api-1; api-2 gets zero traffic until it recovers.")

Notice the difference the layer makes. The L4 balancer sends two connections from the same client (ports 51000 and 51001) to different backends, it balances connections, not requests, and knows nothing about what's inside. The L7 balancer splits traffic by path into separate pools and, crucially, routes every request around the backend that failed its health check. That health check is the heart of availability: the LB probes each backend continuously and, the moment one stops answering, stops sending it traffic, so a crash or a bad deploy takes a server out of rotation in seconds with no user seeing an error.

Choosing a backend, and keeping sessions

Within a healthy pool, the LB needs a policy. Round-robin cycles through backends; it's fine when requests cost about the same. Least-connections sends each new request to the backend with the fewest in flight, which handles uneven request costs far better and is a good default. Power of two choices, which the distributed-systems volume covered, picks two backends at random and takes the less loaded, getting most of least-connections' benefit without global state. For caches and sharded backends, consistent hashing maps each key to a backend so that adding or removing one backend remaps only a small fraction of keys.

A recurring wrinkle is session affinity ("sticky sessions"): routing a given user to the same backend so in-memory session state is there. It works, but it undermines balancing (a popular user overloads one backend) and breaks when that backend dies (the session's gone). The better answer is almost always to make backends stateless, holding session state in a shared store (chapter on caching in the data volume, or a database), so any backend can serve any request and the LB is free to balance purely on load. Statelessness is what makes horizontal scaling and painless failover possible.

The method: localise before you fix

The commonest mistake in debugging a network problem is to guess a cause and start changing things. The disciplined approach is the one this whole volume has been building toward: the stack is layered, so diagnose by layer. A request's life is name → connection → TLS → request → response (the chapters in order), and a failure or slowness lives at exactly one of those. Find which first; the fix follows from the layer.

The single most useful trick is to ask where the time goes. curl will tell you, phase by phase, and because the phases are the layers, the breakdown points straight at the culprit.

# curl reports cumulative timing checkpoints with -w. Each is seconds since the
# request began, so the TIME IN EACH PHASE is the gap between checkpoints. This
# is the single most useful network-diagnosis trick: it tells you WHICH LAYER
# is slow, so you debug the right thing instead of guessing.
#
#   curl -w "@curl-format.txt" -o /dev/null -s https://example.com
#
# with a format file printing time_namelookup, time_connect, time_appconnect,
# time_starttransfer and time_total.

PHASES = [
    # (label, which layer, which tool to reach for next)
    ("DNS lookup",      "naming (ch15)",     "dig / resolvectl"),
    ("TCP connect",     "transport (ch9)",   "traceroute / ss"),
    ("TLS handshake",   "security (ch6)",    "openssl s_client"),
    ("server thinking", "the application",   "server logs / APM"),
    ("download",        "the response/path", "check payload size, bandwidth"),
]

def breakdown(t_dns, t_connect, t_appconnect, t_ttfb, t_total):
    # cumulative checkpoints -> per-phase durations
    durs = [
        t_dns,                    # DNS
        t_connect   - t_dns,      # TCP handshake
        t_appconnect - t_connect, # TLS handshake
        t_ttfb      - t_appconnect, # waiting for first byte (server work)
        t_total     - t_ttfb,     # downloading the body
    ]
    return durs

def diagnose(name, *checkpoints):
    durs = breakdown(*checkpoints)
    total = checkpoints[-1]
    print(f"{name}  (total {total*1000:.0f} ms)")
    worst_i, worst = 0, -1.0
    for i, (dur, (label, layer, tool)) in enumerate(zip(durs, PHASES)):
        pct = dur / total * 100
        bar = "#" * round(pct / 5)
        print(f"  {label:16} {dur*1000:6.0f} ms  {pct:4.0f}%  {bar}")
        if dur > worst:
            worst_i, worst = i, dur
    label, layer, tool = PHASES[worst_i]
    print(f"  -> dominated by {label}: look at {layer}; next tool: {tool}\n")
    return label

# Scenario A: a slow page. Where's the time? (cumulative seconds)
a = diagnose("A: 'the site is slow'", 0.004, 0.028, 0.072, 0.840, 0.880)
# Scenario B: same site, different user, 'slow' again -- but a different cause.
b = diagnose("B: 'same site, also slow'", 0.310, 0.335, 0.380, 0.405, 0.430)
# Scenario C: fast first byte, slow overall -- it's the download, not the server.
c = diagnose("C: 'slow to finish'", 0.005, 0.020, 0.050, 0.070, 1.900)

assert a == "server thinking"   # app is slow -> look at the server, not the network
assert b == "DNS lookup"        # 310ms resolving -> a DNS problem, not the app
assert c == "download"          # server was fast; it's a fat payload or a slow path
print("same symptom ('slow'), three different layers -- the breakdown tells them apart.")

Three users all say "it's slow". The timing breakdown shows they have three unrelated problems: one is the application taking 768ms to produce a byte (look at server logs, not the network); one is spending 310ms in DNS (a resolver problem, chapter 15); one has a fast server but a 1.8-second download (a fat payload or a congested path). Without the breakdown you'd guess; with it, you know which layer to open next. This is not a toy, it's exactly what curl gives you in real life:

$ curl -sS -o /dev/null \
    -w 'dns=%{time_namelookup}  connect=%{time_connect}  tls=%{time_appconnect}  ttfb=%{time_starttransfer}  total=%{time_total}\n' \
    https://example.com

dns=0.049813  connect=0.125818  tls=0.214593  ttfb=0.297702  total=0.298189
# per-phase: DNS 50ms, TCP 76ms, TLS 89ms, server 83ms, download <1ms
# http=2  code=200  ip=2606:4700:10::ac42:93f3   (HTTP/2, over IPv6)

A tool for each layer

Each layer has a tool that answers "is it you?". Learn the one question each answers, not every flag:

Two reach deeper when the high-level tools aren't enough. curl -v shows the whole exchange, DNS result, the IP it connected to, the TLS certificate, every request and response header, which resolves most "why is this response weird?" questions on its own. And when you genuinely need the ground truth, tcpdump (or Wireshark) captures the actual packets, the only tool that can't lie to you, because it shows what crossed the wire rather than what some layer says happened. Reach for it when the layers disagree: the app says it sent the request, the server says it never arrived, tcpdump on both ends settles it. The order is the lesson: start high and cheap (curl -w, DevTools), go low and expensive (tcpdump) only when you must.

Read the DevTools waterfall

In the browser, the Network panel is this same breakdown made visual: each request is a bar split into DNS, connect, TLS, "waiting" (TTFB) and download, exactly the phases above. A long "waiting" segment is a slow server; a long "download" is a fat response; a wall of requests stacked in sequence (rather than overlapping) is a protocol or dependency problem (chapter 5's head-of-line blocking, or render-blocking resources). The waterfall is where the whole volume becomes visible in one screen, and learning to read it is the most transferable network skill a web engineer has.

The whole journey, in order

Here is the request that this volume has been assembling, chapter by chapter, now run start to finish. Each step is tagged with the chapter that explains it, so this doubles as the index to everything before it.

Counting the cost, cold and warm

Narrating the steps is half the answer; the senior half is knowing what each step costs. Almost every step is a round trip, and a round trip's price is distance (chapter 16). So the same journey costs wildly different amounts depending on whether you've been here before.

# What actually happens when you type a URL and press Enter? Every chapter of
# this volume, in order, as one request. Here it is as a timeline, each step
# tagged with the round trips it costs -- because round trips x distance is the
# whole latency story (ch16). We price it two ways: a COLD first visit (every
# handshake from scratch) and a WARM repeat (connection reuse, DNS cached).

RTT_MS = 40  # a realistic round trip to a nearby CDN edge (ch16, ch17)

# (step, chapter, round trips when cold, round trips when warm)
STEPS = [
    ("Parse the URL into scheme/host/path",      "ch2",  0, 0),
    ("Resolve the host with DNS",                "ch15", 1, 0),  # cached on repeat
    ("Open a TCP connection (3-way handshake)",  "ch9",  1, 0),  # reused on repeat
    ("TLS 1.3 handshake",                        "ch6",  1, 0),  # resumed/0-RTT on repeat
    ("Send the HTTP request, get first byte",    "ch3",  1, 1),  # the one unavoidable trip
    ("Server does its work (DB, etc.)",          "ch18", 0, 0),  # not a network trip
    ("Download HTML; multiplex sub-resources",   "ch5",  1, 1),  # H2/H3: one more, overlapped
    ("Browser parses and paints",                "ch1",  0, 0),
]

def total(which):
    i = 2 if which == "cold" else 3
    trips = sum(s[i] for s in STEPS)
    return trips, trips * RTT_MS

print(f"typing https://shop.example.com/cart  (RTT = {RTT_MS} ms)\n")
print(f"{'step':42} {'chapter':7} {'cold':>6} {'warm':>6}")
for name, ch, cold, warm in STEPS:
    print(f"{name:42} {ch:7} {cold:>5}rt {warm:>5}rt")

ct, cms = total("cold")
wt, wms = total("warm")
print(f"\ncold first visit: {ct} round trips  ~= {cms} ms of network latency")
print(f"warm repeat:      {wt} round trips  ~= {wms} ms  (DNS cached, connection reused, TLS resumed)")
print(f"the optimisations in this book removed {ct-wt} of {ct} round trips -- a {cms/wms:.1f}x cut in latency.")

# The single most important fact: almost every step is a ROUND TRIP, and a round
# trip's cost is distance (ch16). You make a page fast by making fewer of them
# (keep-alive, H2/H3 multiplexing, caching, ch3-5) over shorter ones (CDN
# edges and anycast, ch16-17). That's the whole volume in one sentence.
assert (ct, cms) == (5, 200)
assert (wt, wms) == (2, 80)
print("\nfewer round trips, over shorter distances. That is the entire game.")

The cold first visit pays for everything: DNS, TCP, TLS, then the request and its sub-resources, five round trips, 200ms of pure network latency before anything useful. The warm repeat skips three of them, DNS is cached (chapter 15), the connection is reused (chapter 9, keep-alive), TLS is resumed or 0-RTT (chapter 6), leaving just the two round trips to fetch content. Every optimisation in this volume is an attack on that count: keep-alive and multiplexing remove connection round trips (chapters 3, 5); caching removes whole requests (chapter 4); CDNs and anycast shrink the distance each remaining round trip travels (chapters 16, 17); HTTP/3 folds the transport and TLS handshakes together (chapter 5). They are all the same move: fewer round trips, over shorter distances.

The volume in one breath

If you can tell this story, you understand the web's networking. A URL names a resource (ch2) reached over a layered stack where each layer wraps the one above (ch1). DNS turns the name into an address (ch15); packets find their way there across networks joined by routers, through NAT, over IP (ch13, ch14). TCP builds a reliable ordered byte stream on top of unreliable packets with handshakes, ACKs and congestion control (ch9, ch10); TLS makes it private and authenticated (ch6). On that channel, HTTP carries requests and responses (ch3), made fast by caching (ch4) and by multiplexing over HTTP/2 and HTTP/3 (ch5), and made live by WebSockets and the rest (ch7). Scale comes from CDNs, anycast and load balancers putting copies close and spreading the load (ch16, ch17, ch18); and when it breaks, you localise the fault to one layer and fix it there (ch19). All of it in the time it takes to blink, if you've kept the round trips few and the distances short.