Handling more users · Chapter 3

CDNs

How a content delivery network puts copies of your files near users: edges and origins, cache headers, purging and versioned names, origin shields, and what not to cache.

IntermediateInterview priority: High35 min
On this page
  1. The problem: your server is in Virginia, your users are everywhere
  2. The idea: keep copies near the users
  3. How it works: who decides what is cached, and for how long?
  4. What belongs on a CDN
  5. When the file changes: purge or rename
  6. Options and trade-offs
  7. When not to use a CDN
  8. In the interview
  9. Quiz

The problem: your server is in Virginia, your users are everywhere

Your shop's servers sit in one data centre in the eastern United States. For a shopper in New York the home page appears almost instantly. For a shopper in Mumbai or Sydney, every byte crosses an ocean, and a round trip (there and back) takes about 150–250 ms before your server does any work at all.

Far-away users wait for every file, and pages need dozens of files

  1. Distance matters. A server answers nearby users in a few milliseconds, but someone on the other side of the world waits while every byte crosses oceans and continents.
  2. Big files make it worse. A single page can need dozens of images, scripts and fonts, and every one of them makes that long trip.

It gets worse than one round trip. A new HTTPS connection needs a TCP handshake (one round trip) and a TLS handshake (one more with TLS 1.3) before the first request (a third). At 200 ms per round trip that is 600 ms before the first byte of the page arrives. Then the page asks for 60 images, scripts, stylesheets and fonts. Modern browsers fetch many of them over one connection at once, but each still waits at least a round trip, and some can only be discovered after others arrive. A page that loads in 0.5 s in New York takes 3–4 s in Sydney.

Your servers suffer too: every one of those files, for every visitor in every country, is served by your machines and paid for as outbound bandwidth.

The idea: keep copies near the users

Edges in many cities keep copies; the first request is a miss

  1. A CDN, or content delivery network, keeps copies of those files on edge servers in cities all around the world, close to the people using them.
  2. The first request in a region is a miss: the edge fetches the file from your main server, called the origin, and keeps a copy. Everyone after that gets it from nearby, in a few milliseconds.

A CDN (content delivery network) is a network of edge servers in hundreds of cities. The places where a CDN has servers are called points of presence (PoPs). Your own servers become the origin: the source of truth that edges fetch from.

A CDN is a cache spread around the world. When a user in Sydney asks for logo.png:

  1. The request goes to the nearest edge, perhaps 10–20 ms away.
  2. Miss: the edge doesn't have it yet, so it fetches it from the origin (one slow trip), stores a copy and returns it.
  3. Hit: every later request for that file in that region is answered by the edge in a few milliseconds, and never reaches you.

The handshakes now happen with the nearby edge too: 3 round trips of 15 ms is 45 ms instead of 600 ms.

How does the user reach the nearest edge?

Two techniques, often combined:

  • DNS steering: your domain (static.shop.com) is an alias (a DNS CNAME record) for a name the CDN controls. The CDN's DNS servers answer with the address of an edge near the user, based on where the DNS query came from.
  • Anycast: the CDN announces the same IP address from every PoP. The internet's routing system (BGP) then delivers each user's packets to the closest PoP in network terms. No clever DNS needed, and it also spreads out attack traffic.

How it works: who decides what is cached, and for how long?

Mostly you do, with HTTP response headers from the origin. The edge obeys them (unless you override them in the CDN's settings).

HTTP/1.1 200 OK
Content-Type: text/javascript
Cache-Control: public, max-age=31536000, immutable
ETag: "3f9c2a"
HeaderMeaning
Cache-Control: max-age=NAnyone may keep this for N seconds
s-maxage=NOverrides max-age for shared caches (the CDN) only, so the CDN can cache longer than browsers
public / privateprivate means only the user's own browser may cache it, never the CDN
no-storeNobody may cache it at all: use for anything personal or sensitive
no-cacheMay be stored, but must be revalidated with the origin before each use
stale-while-revalidate=NAfter expiry, serve the old copy for up to N seconds while fetching a fresh one in the background
immutableThis URL's content will never change: don't even revalidate
ETag / Last-ModifiedA version tag. When a copy expires the edge asks "still 3f9c2a?" and the origin can reply 304 Not Modified with no body

The cache key is what the edge uses to decide whether two requests are "the same file". By default it is roughly the host plus the URL path and query string. You can add headers to it (for example Accept-Encoding, so gzip and brotli versions are cached separately, via the Vary header) or remove noise from it (marketing query parameters like ?utm_source=), which raises the hit ratio.

Protecting the origin

The origin only sees the misses

  1. This also takes a huge load off your own servers. The origin only sees the occasional miss instead of every single request from every country.

The offload is the share of requests (or bytes) the CDN answers without the origin. For static files 90–99% is normal.

There is a catch: with 300 PoPs, each one misses separately, so a new file can cause 300 origin fetches. Two features fix this:

  • Origin shield (tiered caching): edges fetch from a single regional "shield" cache instead of the origin; only the shield talks to you.
  • Request collapsing: if 1,000 users ask a PoP for the same missing file at once, the PoP sends one request to the origin and makes the rest wait for it.

Worked example: what the CDN saves

A shop serves 50 million page views a day, each pulling about 2 MB of images, scripts and styles.

  • Bytes per day: 50 M × 2 MB = 100 TB.
  • Average bandwidth: 100 TB ÷ 86,400 s ≈ 1.16 GB/s ≈ 9 Gbps. With a peak of 3× the average, about 28 Gbps. Serving that from one data centre would need a lot of network capacity, and far-away users would still be slow.
  • With a 95% byte hit ratio, the origin serves 5%: 5 TB a day, about 1.4 Gbps at peak. A handful of servers manage that.
  • Requests: if each page view makes 60 requests, that is 3 billion a day, ≈ 35,000 per second on average. At 95% offload the origin sees about 1,700 per second.

Latency for distant users drops from 3–4 s to well under 1 s, and the origin's bandwidth bill falls by about 20×. CDN bandwidth costs a few cents per GB or less at volume, usually cheaper than cloud egress from your own servers.

What belongs on a CDN

Same for everyone from the edge; personal data from the origin

  1. CDNs are perfect for static content: files that are the same for everyone, like logos, product photos and videos. Personal things, like your shopping cart, still come from your own servers.
ContentCache at the edge?How
Images, video segments, fontsYes, for a long timeLong max-age, versioned names
JavaScript and CSS bundlesYes, "forever"Content hash in the file name + immutable
HTML for anonymous visitors (articles, product pages)Yes, brieflys-maxage of 30–300 s + purge on edit
Public API responses (prices list, weather)SometimesA few seconds ("micro-caching") absorbs spikes
Account pages, baskets, anything after loginNoCache-Control: private or no-store

Even uncached, personal requests benefit from going through the CDN: the TLS handshake happens at the nearby edge, and the edge keeps warm, long-lived connections to the origin over the CDN's own network ("dynamic acceleration"). Many CDNs can also run small pieces of code at the edge (redirects, A/B tests, authentication checks).

When the file changes: purge or rename

Edges keep serving the old copy until you purge it or rename the file

  1. When a file changes, edges may keep serving the old copy until it expires. Teams either purge the CDN, telling every edge to drop it, or give each new version a new name, like logo dot v2.
ApproachHowGood atWeak atUse when
Wait for expiryShort max-age, do nothingZero effortUsers see old content for up to the TTL; short TTLs cut the hit ratioContent where a minute of staleness is fine
PurgeCall the CDN's API to drop a URL (or a tag) at every edgeSame URL, fresh everywhere within seconds to a few minutesBrowsers that already cached it still hold the old copy; purging a lot at once sends a wave of misses to the originHTML and API responses that editors change
Versioned namesapp.3f9c2a.js, logo.v2.png: a new URL for each versionInstant and certain; old and new never mix; lets you cache "forever"Needs a build step and the HTML must point to the new nameAll static assets (the default)

The standard pattern: assets get content-hashed names and a one-year immutable cache; the HTML that references them gets a short TTL (or no-cache) and is purged on deploy.

Your turn: spot the bug

A team serves app.js and styles.css with Cache-Control: max-age=86400 and fixed names. HTML is not cached. After every deploy, some users get a broken page for hours: buttons don't work and the layout is wrong. Then it fixes itself. What is going on, and how should they set things up?

Compare with our answer
  • The new HTML arrives immediately (it isn't cached), but edges and browsers keep the old app.js and styles.css for up to a day. New HTML plus old scripts is a mismatch, so the page breaks.
  • Purging the CDN only helps partly: browsers also cached the files for a day and the CDN can't reach those.
  • Fix: put a content hash in each asset's name (app.3f9c2a.js) and cache it for a year with immutable. Each deploy creates new names, so the new HTML can only ever load the new files.
  • Keep the HTML on a short TTL or no-cache with an ETag, and purge it on deploy.
  • Keep old assets on the origin for a while: a user with old HTML open (or cached) still requests the old hashed names.

A strong answer mentions: HTML and assets out of sync · browser caches can't be purged · content-hashed names + long immutable TTL · short TTL for HTML · don't delete old assets straight away.

Options and trade-offs

OptionHow it worksGood atWeak atUse when
Pull CDNEdges fetch from the origin on a missNo upload step; caches only what is requestedFirst request per region is slow; origin must be up for missesThe default for websites and APIs
Push CDNYou upload files to the CDN's storage ahead of timeNo cold misses; origin can be offlineYou manage uploads and deletes; you pay to store everythingLarge files known in advance (game patches, software releases)
Object storage as originCDN in front of a storage bucket (e.g. S3)No servers to run for static files; huge capacityOnly static filesImages, video, downloads (object storage)
Multi-CDNTwo providers, traffic split by DNSSurvives one CDN's outage; negotiate on priceTwo configurations to keep identical; lower hit ratio per CDNVery large sites where a CDN outage is unacceptable

When not to use a CDN

  • Every user is close to the origin (an internal tool, a service for one city) and traffic is small: the extra moving part buys little.
  • Nearly everything is personal and real-time: a trading screen or a chat stream gains only connection-level benefits.
  • Data must not leave a country or be stored by a third party (some health, government or financial data): edges store copies, so check the CDN's regional controls first.
  • In practice, almost every public website should use one for its static files: it is cheap, and it also absorbs traffic spikes and many attacks.

In the interview

Your turn: a flash sale abroad

An online shop based in Germany runs a 24-hour sale aimed at India. It expects 5 million visitors, each viewing 15 product pages with 8 images of 200 KB each. Product pages show a price and a "only N left" stock count. Decide what goes through the CDN, with which caching rules, and what you would worry about.

Compare with our answer
  • Images: 5 M × 15 × 8 × 200 KB ≈ 120 TB in a day, about 11 Gbps on average and much more at the peak. They must come from edges in India: versioned names, long max-age, an origin shield in front of object storage.
  • Product pages: the same for every visitor except the stock count. Cache the HTML at the edge for 30–60 s, and load the price and stock separately from a small API call (or micro-cache that API for 1–5 s).
  • Checkout, basket, account: never cached (private, no-store), but still routed through the CDN for nearby TLS and warm connections to Germany.
  • Pre-warm: request the sale's images through Indian PoPs before it starts, so the first visitors don't all miss together.
  • Worries: a price change must be purged quickly; the stock count is approximate on the page, but the purchase must check the real stock at the origin; protect the origin from a mass miss with collapsing and stale-while-revalidate.

A strong answer mentions: bandwidth estimate · images with long TTLs and versioned names · short edge TTL for shared HTML · dynamic stock fetched separately · nothing personal cached · pre-warming and origin protection.

CDNs on one page
  • CDN = edge servers in hundreds of cities caching copies of your files; your servers are the origin.
  • Wins: handshakes and downloads happen ~10–30 ms away instead of 150–250 ms; 90–99% of static traffic never reaches you.
  • Users reach the nearest edge by DNS steering or anycast.
  • You control caching with headers: max-age, s-maxage, private, no-store, no-cache + ETag, stale-while-revalidate, immutable.
  • Default pattern: content-hashed asset names cached for a year; HTML with a short TTL, purged on deploy.
  • Never cache personal responses: send private, no-store; make the CDN opt-in by path.
  • Protect the origin: origin shield, request collapsing, serve stale on error; avoid purging everything at once.
  • Sizing: views × page weight ÷ 86,400 = bandwidth; origin load = total × (1 − hit ratio).
Interactive labPractise it: place edges around the world, shield the origin and ship a new logoOpens in the ByteAtlas lab. Your progress is saved in this browser.
Check what stuck

1. A news site's article pages are the same for every anonymous reader, but editors fix typos within minutes of publishing. What caching setup fits best at the CDN?

2. After a deploy, users report broken pages until they hard-refresh. Scripts use fixed names with a one-day max-age. What is the most reliable fix?

3. A site with 300 PoPs releases a new 50 MB game patch, and the origin is overwhelmed in the first minute. What helps most?

4. Which response should never be cached by a shared CDN edge?

5. Pages take 3 seconds to load in Sydney but 0.5 seconds in New York, from a single US origin. Most of the extra time is before the first byte arrives. Why does a CDN help even for pages it does not cache?

6. 40 million page views a day, 2.5 MB each, and a 96% byte hit ratio at the CDN. About how much does the origin serve per day?

7. Attackers request /products?page=1&x=<random number>, and origin load jumps tenfold. What is the cause and the best fix?

Score 60% or more and the lesson is marked complete.

Passing the quiz also marks it complete. Progress is saved in this browser only.