Building a URL Shortener System Design

Khimananda Oli 8 min read Web Development
Building a URL Shortener System Design

By Khimananda Oli | Last reviewed: August 2026

Building a URL Shortener System Design is a classic engineering challenge because it appears simple but exposes deep distributed systems trade-offs at scale. While generating a short link is trivial for a single server, handling billions of redirects with low latency requires careful choices around ID generation, caching layers, and database partitioning. This guide moves past high-level theory to the concrete configurations and architectural decisions you need to implement a production-grade service in 2026.

How do you architect the core components when building a URL Shortener System Design?

The foundation of any reliable shortener is separating the write path (link creation) from the read path (redirection). These two workflows have fundamentally different consistency and latency requirements. When deciding between monolith and microservices, I recommend starting with a modular monolith where these paths are distinct modules; only split them into separate services when your write volume exceeds 10K/s or your read cache hit ratio drops below 95%.

ClientLoad BalancerAPI Server(Write + Read)Redis Cache(Hot Redirects)Database(Persistent Store)
Core architecture for building a URL shortener system design: separate read and write paths with dedicated caching layer

Your API server must handle two distinct endpoints. The POST /api/v1/shorten endpoint validates the long URL, generates a unique short code, persists the mapping, and returns the result. This is a write-heavy operation that demands strong consistency. The GET /{shortCode} endpoint handles redirection. This path is extremely read-heavy and should tolerate eventual consistency for analytics updates, but requires strict consistency for the redirect target itself. Never perform database writes on the redirect path; use asynchronous event queues for click tracking to keep p99 latency under 50ms.

Choosing the right data model

A common mistake is over-engineering the schema early. For PostgreSQL or MySQL, a minimal table suffices:

CREATE TABLE url_mappings (
    id BIGINT PRIMARY KEY,
    short_code VARCHAR(10) NOT NULL UNIQUE,
    long_url TEXT NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW(),
    expires_at TIMESTAMPTZ,
    owner_id UUID
);

CREATE INDEX idx_short_code ON url_mappings(short_code);
CREATE INDEX idx_expires_at ON url_mappings(expires_at) WHERE expires_at IS NOT NULL;

Note the partial index on expires_at. Most links never expire, so indexing all rows wastes space. If you are evaluating storage engines, my comparison of MariaDB vs MySQL covers performance nuances relevant to this write-once-read-many workload. For NoSQL approaches, DynamoDB or Cassandra work well if you partition by short_code, but relational databases with proper caching often outperform them for this specific access pattern due to simpler operational overhead.

What is the best ID generation strategy for a URL shortener?

ID generation is the most critical decision when building a URL shortener system design. You need unique, compact, URL-safe identifiers without coordination bottlenecks. Avoid UUIDs; their 128-bit entropy produces strings too long for "short" URLs. Instead, use Base62 encoding (a-z, A-Z, 0-9) which yields 62^N combinations. A 7-character Base62 string provides 3.5 trillion unique values—sufficient for most global-scale services.

DB Auto-Increment✓ Simple, collision-free✗ Single point of failure✗ Predictable enumerationHash + Truncate✓ Stateless generation✗ Collision resolution needed✗ Non-sequential storagePre-Allocated Ranges✓ No coordination per request✓ Horizontally scalable✓ Unpredictable to usersBest for: <1K writes/secBest for: Custom slugsBest for: >10K writes/secRecommended: Key Generator ServiceAllocates ranges (e.g., 1M IDs) to API servers via gRPCServers generate locally until range exhausted → zero DB calls per link
ID generation strategy comparison for building a URL shortener system design at varying scales

Implementing the Key Generator Service

For production scale, deploy a dedicated Key Generator Service (KGS) that pre-allocates ID ranges. Each API server requests a batch (e.g., 1 million IDs) and serves them from local memory until exhausted. This eliminates database round-trips during link creation entirely.

// Pseudocode for KGS range allocation
// Database table: key_ranges (id, start_val, end_val, assigned_to, assigned_at)

func AllocateRange(serverID string, batchSize int64) (int64, int64) {
    tx := db.Begin()
    // Atomically claim next available range using SELECT FOR UPDATE SKIP LOCKED
    row := tx.QueryRow(`
        UPDATE key_ranges 
        SET assigned_to = $1, assigned_at = NOW() 
        WHERE assigned_to IS NULL 
        ORDER BY start_val ASC 
        LIMIT 1 
        RETURNING start_val, end_val
    `, serverID)
    
    var start, end int64
    row.Scan(&start, &end)
    tx.Commit()
    return start, end
}

// On API server: convert allocated integer to Base62
func EncodeBase62(num int64) string {
    chars := "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
    result := make([]byte, 0, 7)
    for num > 0 {
        result = append(result, chars[num % 62])
        num /= 62
    }
    // Reverse and pad to fixed length
    reverse(result)
    return string(result)
}

This approach gives you horizontal scalability without coordination overhead. If an API server crashes with unused IDs, those IDs are simply lost—a perfectly acceptable trade-off given the vast Base62 keyspace. For deeper database tuning to support the KGS backing store, see my MySQL performance tuning guide.

How do you optimize caching and redirection performance?

Redirect traffic is typically 100x–1000x higher than creation traffic. Your cache layer determines whether your system survives viral spikes. Use Redis as your primary cache with a TTL matching your link expiration policy. For links without expiration, use a 24-hour TTL with lazy refresh.

Caching StrategyHit RatioLatency ImpactComplexityBest For
Cache-Aside (Lazy Load)95–99%+2ms on missLowGeneral purpose, safe default
Read-Through98–99.5%Consistent lowMediumPredictable latency SLAs
Write-Through99%++5ms on writeHighStrong consistency required
Multi-Tier (L1 Local + L2 Redis)99.5%+<1ms on L1 hitHigh>100K RPS per node

For most teams, cache-aside is the correct starting point. Implement it defensively:

func GetRedirect(shortCode string) (string, error) {
    // Check cache first
    cached, err := redis.Get(ctx, "url:"+shortCode).Result()
    if err == nil {
        return cached, nil
    }
    
    // Cache miss: query database
    mapping, err := db.GetByShortCode(shortCode)
    if err != nil {
        return "", ErrNotFound
    }
    
    // Populate cache with TTL
    ttl := 24 * time.Hour
    if mapping.ExpiresAt != nil {
        ttl = time.Until(*mapping.ExpiresAt)
    }
    redis.Set(ctx, "url:"+shortCode, mapping.LongURL, ttl)
    
    return mapping.LongURL, nil
}

Add a local in-process cache (like Ristretto or BigCache in Go) as L1 for hot keys. This eliminates network serialization for frequently accessed links. Monitor your cache hit ratio as a golden signal; if it drops below 95%, investigate cache eviction policies or increase Redis memory immediately.

How do you handle database scaling and analytics separation?

Your redirect database and analytics database have opposing optimization goals. Redirects need fast point lookups with strong consistency. Analytics needs high-throughput writes and complex aggregations. Never mix these workloads in the same database instance.

API ServerRedirect DB(PostgreSQL / DynamoDB)Event Queue(Kafka / SQS)Analytics Store(ClickHouse / BigQuery)Redis Cache(Read Path Only)Critical: Redirect path never touches analytics store. Async events only.
Separation of concerns: redirect storage vs analytics pipeline for scalable URL shortener system design

Sharding the redirect database

When your mapping table exceeds 100 million rows or 50GB, shard by short_code hash. Consistent hashing distributes load evenly and simplifies resharding. Application-level sharding is preferred over database-native sharding for predictable routing:

  • Compute shard = hash(short_code) % N where N is your shard count
  • Maintain a shard map in configuration or etcd for dynamic rebalancing
  • Route queries directly to the correct shard—no scatter-gather
  • Pre-warm new shards before migration to avoid cold-cache penalties

For analytics, stream click events to Kafka or SQS asynchronously. Consumers batch-write to ClickHouse or BigQuery. This decouples redirect latency from analytics throughput completely. If you need to set up replication for the redirect database's high availability, refer to my guide on PostgreSQL replication and high availability for battle-tested configurations.

Final recommendations for production deployment

Building a URL Shortener System Design successfully means resisting premature complexity while planning for genuine scale points. Start with a single PostgreSQL instance, Redis cache, and the Key Generator Service. Add sharding only when monitoring proves necessity. Instrument everything with OpenTelemetry from day one—redirect latency percentiles and cache hit ratios are non-negotiable observability metrics. Security matters equally: validate all input URLs against SSRF attacks, implement rate limiting per IP and API key, and never expose internal IDs in short codes. If you are designing this system for a team or preparing for production deployment, reach out to discuss architecture review or implementation support tailored to your traffic profile and compliance requirements.

Frequently Asked Questions

PostgreSQL with read replicas handles metadata and analytics well, while Redis caches hot mappings. For pure key-value lookups at massive scale, DynamoDB or ScyllaDB offers lower latency than relational options for the core redirect path.

Use base62 encoding on auto-increment IDs or distributed snowflake generators. Pre-generate batches in Redis to avoid real-time coordination overhead. Avoid random hashing alone as collision probability increases significantly with volume, requiring expensive retry logic during high-write periods.

Use 302 temporary redirects to preserve analytics tracking and allow destination updates. Reserve 301 permanent redirects only for verified, immutable links where caching at the browser level is desired and future link modification is strictly unnecessary.

Expect fifty to one hundred dollars monthly using cloud-native serverless functions and managed caching. Costs scale primarily with cache hit ratios and data transfer egress rather than compute, making CDN integration critical for maintaining predictable operational expenses at this traffic volume.

Implement real-time scanning via Google Safe Browsing API and VirusTotal before activation. Block known disposable domains, enforce HTTPS-only destinations, and apply rate limiting per IP. Maintain an abuse report endpoint for manual review of flagged content to protect platform reputation.

Yes, but generic domains suffer lower click-through rates and higher spam filter blocking. Custom domains improve brand trust, deliverability, and SEO attribution. Most production system designs assume custom domain support for proper analytics segmentation and user confidence in shared links.

Caching stores resolved long URLs in memory, eliminating database lookups for repeat requests. With typical 80/20 access patterns, Redis or CDN edge caching reduces origin load by over ninety percent, cutting p99 latency from milliseconds to microseconds for popular links.

Base62 with seven characters provides 3.5 trillion combinations, sufficient for decades at moderate volume. Monitor namespace utilization proactively and plan migration to longer codes or additional character sets well before exhaustion. Shard ID generation early to enable seamless horizontal scaling.

Store anonymized, aggregated metrics only. Avoid logging IPs or personal identifiers directly. Use cookieless tracking, respect Do Not Track headers, and provide clear privacy policies. Process geolocation and referrer data server-side before discarding raw request details to maintain compliance.

Yes, serverless functions handle spiky redirect traffic efficiently with zero idle cost. Cold starts are acceptable since cached responses return instantly. Pair with edge computing platforms like Cloudflare Workers to execute redirect logic at the network edge, minimizing global latency.

Return a branded 404 page with search functionality instead of generic errors. Log deletion events for audit trails and optionally redirect to a fallback landing page. Set TTL metadata at creation time and run periodic cleanup jobs to purge stale records from storage.

No.

Export mappings via batch ETL, validate checksums post-import, and run dual-write during transition. Use DNS-level cutover with low TTL to minimize downtime. Maintain legacy lookup fallback for thirty days to catch stragglers before decommissioning old infrastructure completely.

Only for async tasks like analytics ingestion, malware scanning, or notification dispatch. The redirect path itself must remain synchronous and stateless. Introduce Kafka or SQS solely for decoupling non-critical workloads to preserve sub-millisecond response times on the hot path.

Track redirect latency p99, cache hit ratio, error rate, and namespace consumption daily.