Bitly — High-Level Design

Nailing down functional requirements (create a short URL with optional alias & expiry, redirect users) and non-functional ones (low latency, 100M DAU, unique URLs, high availability with eventual consistency). The API surface is a POST /url to create and GET /url/{short_code} to redirect.

The simplest thing that works: a single server writing to a database, returning 301/302 redirects on reads. Good enough for roughly 1–10k users — a sensible starting point before optimizing.

Targeting 100M daily active users works out to ~1000 requests/sec (and bursts far higher). A single vertically-scaled box hits a hardware ceiling — you can't just keep buying a bigger EC2 instance.

Move to horizontally-scaled services behind a load balancer. The url_table stores short_code (indexed), long_url, alias, created_at and user_id — roughly 242 bytes per row, or ~242 GB for 1 billion URLs.

242 GB fits comfortably in a single modern DB, but the real problem is query volume: one viral short link can trigger millions of reads. Caching protects the stateful database — the resource we least want to overload.

Bitly is read-heavy — URLs are created rarely but redirected constantly. Splitting into dedicated Read and Write services lets each scale independently. Reads check Redis first and fall back to the DB.

On a GET, the read service checks Redis; on a miss it queries the DB and populates the cache. This keeps hot links blazing fast and shields the database from the redirect firehose.

The write service owns short-code generation. A naive MD5 hash risks collisions, so a better approach is a global counter fed into the hashing step to guarantee uniqueness.

To generate unique IDs without a single bottleneck, each write server gets a worker ID from Zookeeper and keeps a local counter. A Snowflake-style 64-bit ID (timestamp + worker ID + sequence) is generated per server and hashed into a short code.