← Back to System Internals

Designing a Unique ID Generator

A system design interview guide to handing out identifiers that are unique across every machine in a distributed system, without a central bottleneck, while staying small, numeric, and roughly ordered by time.

Almost every distributed system needs to label things: a new row, a message, an order, an uploaded photo. On a single database the answer is easy — an auto_increment column hands out 1, 2, 3, and the database guarantees no two rows collide. The moment you have many machines minting IDs at once, that guarantee disappears. You cannot ask a single counter for every ID without turning it into a bottleneck and a single point of failure, yet you still need every ID to be globally unique. This guide works through the requirements, compares the common approaches and where they break, and then explains the Snowflake design in detail, including the one subtle failure mode — clock skew — that every time-based scheme has to handle.

1. Requirements

Before choosing a design, pin down what an ID actually has to satisfy. In an interview these constraints are what justify ruling out simpler options, so state them explicitly.

2. Approaches and Their Drawbacks

There are several well-known ways to generate IDs in a distributed setting. Each makes a different trade, and walking through why the first three fall short is the cleanest way to motivate the fourth.

The pattern across the first three is a tug-of-war between coordination and independence. Multi-master and the ticket server stay correct by coordinating, which limits scaling and creates bottlenecks. UUIDs avoid coordination entirely but pay for it in size and the loss of ordering. Snowflake threads the needle: it needs no coordination on the hot path, yet still produces compact, ordered, numeric IDs.

3. The Snowflake Approach

The key insight behind Snowflake (originally from Twitter) is to divide and conquer the 64 bits. Instead of treating an ID as one opaque counter, you partition the bits into sections, and let a different source of uniqueness fill in each section. As long as the combination of all sections is unique, the whole ID is unique — and crucially, each section can be filled in locally, with no call to a central server.

Reading the layout from the most significant bit to the least:

• The sign bit (1 bit) is reserved and always 0, so every ID is a positive number.
• The timestamp (41 bits) sits in the high-order positions. Because it is the most significant field, IDs naturally sort by time: a later timestamp produces a numerically larger ID.
• The data center ID (5 bits) and machine ID (5 bits) together identify the exact node that produced the ID. They make IDs from different nodes unique even when those nodes generate at the same millisecond.
• The sequence number (12 bits) distinguishes IDs produced by the same node within the same millisecond.

Uniqueness comes from the combination. Two IDs differ if they were made at different times (timestamp differs), or on different nodes (data center / machine ID differ), or on the same node in the same millisecond but at a different point in that millisecond (sequence differs). There is no way for two valid IDs to share all four fields.

# generating one ID on a node, no network call
function next_id():
  ts = current_millis()
  if ts == last_ts:
    sequence = (sequence + 1) AND 0xFFF   # 12-bit wrap (0..4095)
    if sequence == 0:
      ts = wait_until_next_millis(last_ts) # exhausted this ms
  else:
    sequence = 0
  last_ts = ts
  return ((ts - CUSTOM_EPOCH) << 22)        # 41 bits, shifted up
       OR (datacenter_id << 17)             # 5 bits
       OR (machine_id << 12)                # 5 bits
       OR sequence                          # 12 bits

4. Snowflake Fields and the Math

Each field's bit width sets a concrete limit. Working through the arithmetic shows why these particular sizes are a sensible default, and what each one buys you.

Two numbers are worth saying out loud in an interview. First, the custom epoch: by counting milliseconds from a recent date instead of 1970, you do not waste the 41-bit timestamp range on decades that already passed, so the ~69 years of headroom start from when the system was deployed. Second, the throughput ceiling: 4096 IDs per millisecond per node is just over four million IDs per second per node, and with up to 1024 nodes the system as a whole can mint on the order of four billion IDs per second — far beyond what most systems need.

Putting the fields together, the resulting IDs are roughly monotonic by time and globally unique without coordination. "Roughly" monotonic, not strictly: IDs generated on different nodes in the same millisecond are ordered by node ID rather than by the instant within the millisecond, so two IDs a fraction of a millisecond apart on different nodes may not sort in true wall-clock order. For pagination and range scans this is more than good enough.

5. Clock Skew and Rollback

Snowflake's elegance comes from trusting the local clock, and that trust is also its one weak point. The timestamp field assumes the clock only ever moves forward. In reality, a server's clock can jump backwards — most commonly when NTP (the Network Time Protocol) corrects a clock that had drifted ahead, or after a manual time change. This is clock skew, and if it is ignored it can break uniqueness.

The danger is concrete. Suppose a node mints IDs at millisecond T, then its clock is corrected backwards to T - 5. As it generates again, it will produce timestamps it has already used. Combined with the same node ID and a sequence counter that may also repeat, the node can emit an ID identical to one it produced five milliseconds earlier — a collision, the one thing the system must never do.

The mitigations form a layered defense:

• Refuse to go backwards. The generator remembers the last timestamp it used. If the current clock reading is earlier than that, it knows the clock rolled back and must not emit an ID with the stale timestamp.
• Wait it out. For a small rollback, the simplest safe response is to block — spin or sleep until the clock catches back up to the last timestamp it already issued — and only then resume. No ID is ever issued with a timestamp the node has gone past.
• Fail loudly on a large rollback. If the clock has jumped back by more than a small, tolerable amount, waiting would stall the node for too long. In that case the node should raise an error and stop serving rather than risk a collision or an unbounded pause, letting it be taken out of rotation.
• Keep clocks tight with NTP. Running NTP so clocks stay closely synchronized makes large jumps rare in the first place. Where available, a monotonic clock source (one that, by construction, never goes backwards) can drive the wait logic, sidestepping NTP step corrections entirely.

# guarding against a backward clock
function next_id():
  ts = current_millis()
  if ts < last_ts:                          # clock moved backwards
    if (last_ts - ts) <= MAX_TOLERANCE:
      ts = wait_until(last_ts)              # small skew: block and recover
    else:
      raise ClockMovedBackwardError         # large skew: refuse to serve
  # ... normal sequence + pack as before ...

6. Summary

Designing a unique ID generator is an exercise in getting global uniqueness without global coordination. Snowflake achieves it by partitioning 64 bits so each part can be filled in locally.