Building TinyURL: A Deep Dive into URL Shortener System Design 🚀🔗 #SystemDesign #URLShortener #TechInsights

Photo by Med Badr Chemmaoui on Unsplash

🚨 Problem Statement

Assuming we would like to design a URL shortener service, various key considerations must be addressed. To start with, we need to decide on our service’s functionality.

🎯 Key Considerations

1️⃣ Customization 🛠️

• Will users be able to customize their shortened URLs?
• Alternatively, will the service generate a random string of characters for each URL?

2️⃣ Scalability 📈 & Storage 💾

• How will we store the data associated with each shortened URL?
• Should we use a database? If so, which type would be best suited for our needs?
• How often will the application access data?
• Do we need to support real-time lookups?

3️⃣ User Interaction 🖥️📱

• How will users interact with the service?
• Web interface 🌐
• API ⚡

Depending on our choice, we need to design our system accordingly.

🔑 Summary

Designing a URL shortener service requires careful planning 📝 and consideration 🤔 of various factors. By taking the time to define our requirements upfront, we can create a system that is both scalable ⚙️ and secure 🔐.

🎥 Check out this video below for a detailed discussion on how you should approach this system design interview question:

Introduction

Implementing a tiny URL 🔗 using the minimalistic approach ✨ shortens a long original URL into a shorter and manageable URL.

🔍 What is a URL Shortener?

This project concerns the system design 🏗️ of a URL shortener service like bit.ly, goo.gl, or tinyurl.com. We’ll examine the various design aspects 🔄 and trade-offs ⚖️ in creating such a service.

A URL shortener service is a web service 🌐 that provides shortened URLs for long URLs. The shortened URL is:
✅ Easier to remember 🧠
✅ More convenient to share 📢
✅ Capable of tracking 📊 clicks and analytics

🌎 Existing URL Shortener Services

Many popular URL shortener services exist, such as:
🔹 bit.ly
🔹 goo.gl
🔹 tinyurl.com

📌 In this project, we will design our own URL shortener service! 🏗️🔗

Let’s dive in! 🚀

🔍 Clarifications

To ensure a well-structured URL shortener service, we need to clarify certain requirements:

🔒 Privacy & Access Control

• Can users create private URLs 🔑 that only specific users can access?

📊 Analytics & Tracking

• How many times has a short URL been used? 🔄
• What were the user locations 🌍, devices 📱, and referring websites 🔗?
• How will we store these statistics in our system? 💾

🛠 Customization Options

• Should users be able to choose their own custom URL instead of a randomly generated one? ✏️

📈 Tracking Features

• The service should be able to track the number of clicks on each shortened URL 📊.

🏛 Storage & Access

• We will use a database to store all URL mappings and metadata 💾.
• Data must be accessed in real-time ⏳ to ensure fast redirection.

🛡 Security Measures

• Rate limiting ⏳ to prevent abuse and excessive API requests 🚫.
• Implement filtering mechanisms to prevent malicious URL submissions ⚠️.

⚙️ User Interaction & API Support

• Users will interact with the service primarily via an API ⚡.
• Consider providing a web interface 🌐 for easier usability.

This ensures our service is scalable, secure, and user-friendly 🚀.

📝 User Stories / Functional Requirements

🚀 Minimum Viable Product (MVP)

1️⃣ Basic URL Shortening 🔗

As a user, I can enter a URL to shorten in the system and receive a unique shortened URL.

Details: The system should ensure that no two URLs map to the same shortened URL.

2️⃣ Redirection to Original URL 🔄

As a user, I can enter a shortened URL into my browser and be redirected to the original URL instantly.

🛠 Optional Features

3️⃣ Custom Shortened URLs ✏️

As a user, I can enter a URL along with a custom string so that the system returns a shortened URL with my chosen suffix (if available).

💡 Other Functional User Stories

🔹 Easy Sharing 📢

As a user, I want to shorten a URL quickly so that I can share it easily.

🔹 Click Tracking & Analytics 📊

As a user, I want to view statistics about how many people clicked my shortened URL to track its effectiveness.

🔹 Custom Shortcodes 🔠

As a user, I want to choose my own shortcode for a URL so that I can remember it more easily.

🔹 Security & Protection 🛡

As a user, I want the service to be secure so that my data is not compromised.

🔹 High Speed & Performance ⚡

As a user, I want the service to be fast so that I can shorten and share my URLs instantly.

🔹 Free Service 💰

As a user, I want the service to be free to use so that I don’t have to pay anything.

⚙️ Non-Functional Requirements (NFRs)

For our URL Shortener Service 🔗, we must ensure it is fast, reliable, and resilient 💪. Below are the key non-functional requirements (NFRs):

⚡ 1. Low Latency

• URL redirection 🔄 should happen in real-time ⏳ with minimal latency.
• Users should experience instantaneous redirects when accessing a shortened URL.

🔄 2. High Availability & Fault Tolerance

• The system should be highly available (99.99% uptime) ⏲️.
• It should remain operational even if a server crashes 🚨 due to:
• Power outages ⚡
• Hard disk failures 💾
• Network issues 🌐
• A load balancer 🏗️ should distribute traffic across multiple instances for seamless failover.

🔗 3. Strong Data Consistency

• Once a URL is shortened, it must always resolve consistently ✅.
• Given a shortened URL, the system should always return the same original URL, regardless of when it is requested.

🛡 4. No Downtime / Resilient Architecture

• The service should never go down. 🚫📉
• Even if some machines fail, the system should continue functioning without interruption.
• The only failure scenario would be if all servers go down at once, which is highly unlikely.

Ensuring these NFRs will make the URL Shortener Service scalable, reliable, and efficient 🚀🔗.

📊 Capacity Estimation & Constraints

To ensure our URL Shortener Service 🔗 is scalable and efficient, we need to estimate CPU, storage, bandwidth, and caching requirements.

🖥️ CPU (Compute) Requirements

• The system is read-heavy 📖, with a 100:1 ratio between reads and writes.
• Retention period: URLs will be stored for 5 years.
• Traffic estimates:
• 500 million new URLs 🔗 shortened per month 📅.
• Redirections per month = 100 * 500M = 50 billion 🔄.
• Queries per second (QPS) =
  📌 500 million / (30 days * 24 hours * 3600 sec) = 20,000 requests/sec

💾 Storage Requirements

• Total URLs stored = 500M * 5 years * 12 months = 30 billion URLs 📦.
• Storage per URL (including metadata) = 1 KB.
• Total storage required = 30 billion * 1 KB = 30 TB.

🌐 Network Bandwidth Estimates

Write Requests (Input):

• 200 * 1 KB = **200 KB/s**

Read Requests (Output):

• 20,000 * 1 KB/sec = **20 MB/s**

⚡ Cache Estimation (80–20 Rule)

📌 80% of traffic is generated by 20% of the URLs.

• We use LRU (Least Recently Used) caching 🏗️ with a Linked HashMap to store frequently accessed URLs.

Total cacheable requests per day:

• 20,000 * 3600 sec * 24 hours = 1.7 billion 🔄

Cache storage required:

• 1.7B * 20% * 1 KB = **340 GB**

💡 Since many requests may be duplicates, actual cache usage might be lower.

🌐 API Design for URL Shortener

To enable efficient URL shortening and redirection, we provide a simple RESTful API 🛠️.

🚀 1. Create a Short URL

📌 Endpoint:

POST /api/link HTTP/1.1
Host: urlshrtnr.taruntelang.repl.co
Content-Type: application/x-www-form-urlencoded

📌 Request Body:

{
  "link": "https://www.linkedin.com/in/taruntelang/"
}

📌 Response:

{
  "shortURL": "6U8Ahx9"
}

💡 Example:

• Input: https://www.linkedin.com/in/taruntelang/
• Output: https://urlshrtnr.taruntelang.repl.co/6U8Ahx9

🔗 2. Retrieve the Original URL

📌 Endpoint:

GET /6U8Ahx9 HTTP/1.1
Host: urlshrtnr.taruntelang.repl.co

📌 Response:

{
  "originalURL": "https://www.linkedin.com/in/taruntelang/"
}

💡 Behavior:

• The user enters https://urlshrtnr.taruntelang.repl.co/6U8Ahx9 in the browser.
• The system redirects to the original URL instantly.

🎯 Why Short Links?

✅ Easy to share via messages, tweets, or print media 📝📢.
✅ Reduces clutter in social media posts 🌎.
✅ Tracks analytics for marketing performance 📊.
✅ Hides affiliate links for better engagement 🎭.

This API-first approach ensures a simple yet powerful URL shortener service 🚀🔗.

⚙️ System Components

1️⃣ Application Server 🖥️

• Handles URL shortening, redirections, analytics tracking, and API requests.
• Should be stateless, allowing easy horizontal scaling.

2️⃣ Cache (e.g., Redis / Memcached) ⚡

• Stores frequently accessed URLs to reduce database load.
• Uses LRU (Least Recently Used) eviction policy.

3️⃣ Database (NoSQL — e.g., Cassandra, DynamoDB, MongoDB) 💾

• Scalable and optimized for high read traffic.
• Stores URL mappings and user-related data.
• Replication & Sharding for improved performance.

📊 Telemetry & Analytics

Tracking user interactions helps improve performance and understand user behavior. The system should capture:

📍 User Tracking Data:

• 🌍 Country of the visitor
• 📅 Date & Time of access
• 🔗 Referrer URL (where the user clicked from)
• 🌐 Browser type (Chrome, Firefox, etc.)
• 📱 Platform (Mobile, Desktop, Tablet, etc.)

📌 Use Case:

• 📊 Marketers can analyze link engagement to optimize campaigns.
• 🛡 Helps detect bot traffic or suspicious activity.
• 🚀 Provides insights for scaling infrastructure based on traffic patterns.

🏥 Health Check System

🔹 Ensures that the service remains available and performs optimally 💡.

✅ Periodically checks system health by monitoring:

• Database connectivity 🔄
• Cache performance ⚡
• API response times ⏳
• Server resource utilization (CPU, RAM) 📊

📌 Actionable Insights:

• Auto-restart failing services 🚑
• Alert engineers if an anomaly is detected 🚨
• Log failures for debugging 📝

⚖ Load Balancer

A Load Balancer 🔀 is critical for distributing traffic across multiple servers to:

✅ Prevent server overload 🚦
✅ Ensure high availability (If one server fails, traffic is routed to another)
✅ Optimize response times ⚡

📌 Common Load Balancer Options:

• Software-based: Nginx, HAProxy 🖥️
• Cloud-based: AWS ELB, Google Cloud Load Balancer ☁️

🔹 How It Works:

1. User requests a shortened URL 🔗
2. The load balancer directs the request to an available server 🎯
3. The server fetches the original URL from the database or cache 🏗️
4. The system redirects the user to the original website 🌐

🚀 Final Thoughts

By integrating Telemetry, Health Checks, and Load Balancing, our URL Shortener becomes:
✔ Reliable (Auto-failover, Monitoring)
✔ Scalable (Handles millions of requests per second)
✔ Optimized (Efficient resource utilization)

This modular approach ensures high performance while keeping the system resilient and user-friendly 🔥🔗.

📂 Database Schema

1️⃣ URL Table 🔗

UrlId: NUMBER <PRIMARY KEY>
OriginalURL: VARCHAR
CreationDate: DATETIME
ExpiryDate: DATETIME
UserId: NUMBER <FOREIGN KEY>

2️⃣ User Table 👤
UserId: NUMBER <PRIMARY KEY>
Name: VARCHAR
Email: VARCHAR
CreationDate: DATETIME
LastLoginDate: DATETIME

💡 Why NoSQL?

• We need to store billions of rows 📊.
• Easy horizontal scaling ⚖️.
• Minimal relationships between entities.

🔢 Short URL Generation

• Shortened URL format: 7-character string using Base64 encoding 🔡.
• Character set: [a-z A-Z 0–9 - _] (Base64 URL variant).
• Ensures uniqueness by checking for collisions before storing.

📌 Key Takeaways

✔ NoSQL for scalability 📈.
✔ Cache for performance optimization ⚡.
✔ Stateless architecture for easy scaling 🚀.
✔ 7-character Base64 encoding ensures compact short URLs 🔗.

This setup ensures a fast, scalable, and reliable URL shortening system 🔥.