Redis Data Structures Deep Dive: Mastering the Building Blocks

Explore Redis's rich data types and learn when and how to use each one effectively

Introduction

While many developers think of Redis as just a key-value store, it's actually much more powerful. Redis supports a rich set of data structures that go far beyond simple strings. Understanding these data structures and when to use them is crucial for building efficient, scalable applications.

In this comprehensive guide, we'll explore each Redis data structure in detail, with practical examples and performance considerations. By the end, you'll know exactly which data structure to choose for any given scenario.

Overview of Redis Data Structures

Redis provides seven core data structures, each optimized for specific use cases:

Strings: The simplest data type, perfect for caching and counters
Lists: Ordered collections ideal for queues and timelines
Sets: Unordered collections of unique elements
Sorted Sets: Sets with scores for ranking and leaderboards
Hashes: Field-value pairs, like objects or records
Streams: Append-only logs for real-time data processing
Geospatial: Location-based data with geographic operations

Each data structure has specific commands, memory characteristics, and performance profiles. Let's dive deep into each one.

Strings: The Foundation

What are Redis Strings?

Strings are the most basic Redis data type and the building block for all other structures. A Redis string can store:

Text data (UTF-8 encoded)
Binary data (images, files)
Serialized objects (JSON, XML)
Numbers (for arithmetic operations)

String Commands

Basic Operations

# Set and get values
SET user:1000:name "John Doe"
GET user:1000:name

# Set with expiration (TTL in seconds)
SETEX session:abc123 3600 "user_data"

# Set only if key doesn't exist
SETNX config:maintenance "true"

# Set multiple values atomically
MSET user:1001:name "Jane" user:1001:email "jane@example.com"
MGET user:1001:name user:1001:email

Numeric Operations

# Increment/decrement operations
SET page:views 100
INCR page:views              # Returns 101
INCRBY page:views 10         # Returns 111
DECR page:views              # Returns 110
DECRBY page:views 5          # Returns 105

# Float operations
SET product:price 29.99
INCRBYFLOAT product:price 5.01  # Returns 35.0

String Manipulation

# Append to string
SET message "Hello"
APPEND message " World"      # Returns 11 (length)
GET message                  # Returns "Hello World"

# Get string length
STRLEN message               # Returns 11

# Get substring
GETRANGE message 0 4         # Returns "Hello"
SETRANGE message 6 "Redis"   # Replaces "World" with "Redis"

Common Use Cases for Strings

Caching

# Cache API responses
SET cache:weather:london "{\"temp\":20,\"humidity\":65}" EX 1800

# Cache user sessions
SET session:user123 "logged_in" EX 3600

Counters and Analytics

# Page view counter
INCR page:home:views
INCR page:about:views

# Rate limiting
SET rate_limit:user:123 1 EX 60
INCR rate_limit:user:123

Feature Flags

# Simple feature flags
SET feature:new_ui "enabled"
SET feature:beta_feature "disabled"

Performance Characteristics

Memory: ~40 bytes overhead + data size
Access time: O(1) for all operations
Max size: 512MB per string
Best for: Simple values, counters, caching

Lists: Ordered Collections

What are Redis Lists?

Redis Lists are ordered collections of strings, implemented as linked lists. They maintain insertion order and allow duplicate elements. Lists excel at scenarios where you need to:

Maintain order of elements
Add/remove elements from both ends efficiently
Implement queues, stacks, or timelines

List Commands

Adding Elements

# Add to the left (beginning)
LPUSH messages "Hello"
LPUSH messages "Hi"          # List is now ["Hi", "Hello"]

# Add to the right (end)
RPUSH messages "Goodbye"     # List is now ["Hi", "Hello", "Goodbye"]

# Add multiple elements
LPUSH messages "Hey" "Howdy" # List is now ["Howdy", "Hey", "Hi", "Hello", "Goodbye"]

# Insert before/after specific element
LINSERT messages BEFORE "Hello" "Bonjour"

Retrieving Elements

# Get elements by range (0-based indexing)
LRANGE messages 0 -1         # Get all elements
LRANGE messages 0 2          # Get first 3 elements
LRANGE messages -2 -1        # Get last 2 elements

# Get element by index
LINDEX messages 0            # Get first element
LINDEX messages -1           # Get last element

# Get list length
LLEN messages

Removing Elements

# Remove from ends
LPOP messages                # Remove and return first element
RPOP messages                # Remove and return last element

# Remove specific elements
LREM messages 1 "Hello"      # Remove first occurrence of "Hello"
LREM messages -1 "Hi"        # Remove last occurrence of "Hi"
LREM messages 0 "Hey"        # Remove all occurrences of "Hey"

# Trim list to specific range
LTRIM messages 0 99          # Keep only first 100 elements

Blocking Operations

# Block until element is available (useful for queues)
BLPOP task_queue 30          # Block for 30 seconds waiting for element
BRPOP task_queue 0           # Block indefinitely

# Move element between lists atomically
RPOPLPUSH source_list dest_list

Common Use Cases for Lists

Task Queues

# Producer adds tasks
LPUSH task_queue "process_payment:123"
LPUSH task_queue "send_email:456"

# Consumer processes tasks
BRPOP task_queue 30

Activity Feeds

# Add new activities to user feed
LPUSH user:1000:feed "liked photo:789"
LPUSH user:1000:feed "commented on post:456"

# Get recent activities
LRANGE user:1000:feed 0 9    # Get last 10 activities

# Limit feed size
LTRIM user:1000:feed 0 99    # Keep only last 100 activities

Performance Characteristics

Add/remove from ends: O(1)
Access by index: O(N) - use sparingly
Range operations: O(S+N) where S is start offset, N is number of elements
Memory: ~40 bytes per element overhead
Best for: Queues, activity feeds, recent items

Sets: Unique Collections

What are Redis Sets?

Redis Sets are unordered collections of unique strings. Sets automatically handle uniqueness and provide powerful operations for membership testing, intersections, unions, and differences.

Set Commands

Basic Operations

# Add members to set
SADD user:1000:interests "programming" "music" "travel"
SADD user:1000:interests "programming"  # Won't add duplicate

# Check membership
SISMEMBER user:1000:interests "music"   # Returns 1 (true)
SISMEMBER user:1000:interests "sports"  # Returns 0 (false)

# Get all members
SMEMBERS user:1000:interests

# Get set size
SCARD user:1000:interests

# Remove members
SREM user:1000:interests "travel"

Random Operations

# Get random member(s) without removing
SRANDMEMBER user:1000:interests
SRANDMEMBER user:1000:interests 2

# Remove and return random member(s)
SPOP user:1000:interests
SPOP user:1000:interests 2

Set Operations

# Create sample sets
SADD user:1000:interests "programming" "music" "travel"
SADD user:2000:interests "music" "sports" "cooking"

# Intersection (common elements)
SINTER user:1000:interests user:2000:interests  # Returns "music"

# Union (all unique elements)
SUNION user:1000:interests user:2000:interests

# Difference (elements in first set but not in second)
SDIFF user:1000:interests user:2000:interests

# Store results of set operations
SINTERSTORE common_interests user:1000:interests user:2000:interests

Common Use Cases for Sets

Tags and Categories

# Article tags
SADD article:123:tags "redis" "database" "nosql" "caching"

# Find articles with specific tag
SISMEMBER article:123:tags "redis"

# Get all tags for an article
SMEMBERS article:123:tags

Social Features

# User followers
SADD user:1000:followers "user:2000" "user:3000" "user:4000"

# User following
SADD user:1000:following "user:5000" "user:6000"

# Find mutual followers
SINTER user:1000:followers user:2000:followers

# Check if user follows another
SISMEMBER user:1000:following "user:5000"

Unique Visitors Tracking

# Track unique visitors per day
SADD visitors:2023-12-01 "user:123" "user:456" "user:789"

# Get unique visitor count
SCARD visitors:2023-12-01

# Check if user visited today
SISMEMBER visitors:2023-12-01 "user:123"

Blacklists and Whitelists

# IP blacklist
SADD blacklisted_ips "192.168.1.100" "10.0.0.50"

# Check if IP is blacklisted
SISMEMBER blacklisted_ips "192.168.1.100"

Performance Characteristics

Add/remove/membership test: O(1) average
Set operations: O(N) where N is the size of the smaller set
Memory: ~40 bytes per element overhead
Best for: Unique collections, social graphs, filtering

Sorted Sets: Ranked Collections

What are Redis Sorted Sets?

Sorted Sets (ZSets) combine the uniqueness of Sets with the ordering capability of Lists. Each element has an associated score that determines its position in the sorted order. This makes them perfect for leaderboards, rankings, and priority queues.

Sorted Set Commands

Adding and Updating

# Add members with scores
ZADD leaderboard 100 "player1" 200 "player2" 150 "player3"

# Update scores (add to existing score)
ZINCRBY leaderboard 50 "player1"  # player1 now has score 150

# Add only if member doesn't exist
ZADD leaderboard NX 300 "player4"

# Add only if member exists (update only)
ZADD leaderboard XX 250 "player2"

Retrieving by Rank

# Get members by rank (0-based, lowest to highest score)
ZRANGE leaderboard 0 -1          # All members, ascending
ZRANGE leaderboard 0 2           # Top 3 by rank (lowest scores)

# Get members with scores
ZRANGE leaderboard 0 -1 WITHSCORES

# Reverse order (highest to lowest score)
ZREVRANGE leaderboard 0 2        # Top 3 highest scores
ZREVRANGE leaderboard 0 -1 WITHSCORES

Retrieving by Score

# Get members by score range
ZRANGEBYSCORE leaderboard 100 200
ZRANGEBYSCORE leaderboard 100 200 WITHSCORES

# Use infinity for open ranges
ZRANGEBYSCORE leaderboard 100 +inf
ZRANGEBYSCORE leaderboard -inf 200

# Limit results
ZRANGEBYSCORE leaderboard 100 200 LIMIT 0 10

Information and Removal

# Get member's score
ZSCORE leaderboard "player1"

# Get member's rank (0-based)
ZRANK leaderboard "player1"      # Rank by ascending score
ZREVRANK leaderboard "player1"   # Rank by descending score

# Count members in score range
ZCOUNT leaderboard 100 200

# Get set size
ZCARD leaderboard

# Remove members
ZREM leaderboard "player1"

# Remove by rank
ZREMRANGEBYRANK leaderboard 0 2  # Remove top 3 by rank

# Remove by score
ZREMRANGEBYSCORE leaderboard 0 100

Common Use Cases for Sorted Sets

Gaming Leaderboards

# Update player scores
ZADD game:leaderboard 1250 "player:alice" 1100 "player:bob" 1300 "player:charlie"

# Get top 10 players
ZREVRANGE game:leaderboard 0 9 WITHSCORES

# Get player's rank
ZREVRANK game:leaderboard "player:alice"

# Get players around a specific player
ZREVRANK game:leaderboard "player:alice"  # Get rank first
ZREVRANGE game:leaderboard 8 12  # Get players around rank 10

Priority Queues

# Add tasks with priorities (lower score = higher priority)
ZADD task_queue 1 "critical_backup"
ZADD task_queue 3 "send_newsletter"
ZADD task_queue 2 "process_payments"

# Get highest priority task
ZRANGE task_queue 0 0  # Get task with lowest score
ZREM task_queue "critical_backup"  # Remove after processing

Time-based Data

# Store events with timestamps
ZADD user:events 1671234567 "login" 1671234590 "view_product" 1671234610 "add_to_cart"

# Get events in time range
ZRANGEBYSCORE user:events 1671234560 1671234600

# Get recent events
ZREVRANGE user:events 0 9  # Last 10 events

Rate Limiting

# Sliding window rate limiting
# Add current timestamp for each request
ZADD rate_limit:user:123 1671234567 "req1" 1671234568 "req2"

# Remove old entries (older than 60 seconds)
ZREMRANGEBYSCORE rate_limit:user:123 0 1671234507

# Check current request count
ZCARD rate_limit:user:123

Performance Characteristics

Add/remove/update: O(log N)
Range operations: O(log N + M) where M is the number of elements returned
Score/rank lookup: O(log N)
Memory: ~40 bytes per element + score overhead
Best for: Leaderboards, priority queues, time-series data

Hashes: Object-like Structures

What are Redis Hashes?

Redis Hashes are maps between string fields and string values, similar to objects in programming languages or rows in a database. They're perfect for representing objects and provide memory-efficient storage for related data.

Hash Commands

Setting and Getting Fields

# Set individual fields
HSET user:1000 name "John Doe" email "john@example.com" age "30"

# Set multiple fields at once
HMSET user:1001 name "Jane Smith" email "jane@example.com" age "25" city "New York"

# Get individual field
HGET user:1000 name

# Get multiple fields
HMGET user:1000 name email age

# Get all fields and values
HGETALL user:1000

# Set field only if it doesn't exist
HSETNX user:1000 phone "123-456-7890"

Field Operations

# Check if field exists
HEXISTS user:1000 email

# Get all field names
HKEYS user:1000

# Get all values
HVALS user:1000

# Get number of fields
HLEN user:1000

# Delete fields
HDEL user:1000 age phone

Numeric Operations

# Increment field value
HSET user:1000 login_count 5
HINCRBY user:1000 login_count 1    # Now 6
HINCRBYFLOAT user:1000 balance 10.50

Common Use Cases for Hashes

User Profiles

# Store user information
HSET user:1000 name "John Doe" email "john@example.com" created_at "2023-01-15" status "active"

# Update specific fields
HSET user:1000 last_login "2023-12-01T10:30:00Z"

# Get user profile
HGETALL user:1000

Shopping Carts

# Add items to cart (field = product_id, value = quantity)
HSET cart:user123 product:456 2 product:789 1 product:321 3

# Update quantity
HINCRBY cart:user123 product:456 1  # Increase quantity by 1

# Get cart contents
HGETALL cart:user123

# Remove item from cart
HDEL cart:user123 product:789

Configuration Settings

# Application settings
HSET app:config max_connections 100 timeout 30 debug_mode "false"

# Get specific setting
HGET app:config max_connections

# Update setting
HSET app:config debug_mode "true"

Product Information

# Store product details
HSET product:123 name "Laptop" price "999.99" category "Electronics" stock "25"

# Update stock
HINCRBY product:123 stock -1  # Decrease stock by 1

# Get product info
HMGET product:123 name price stock

Performance Characteristics

Field operations: O(1) for single field, O(N) for N fields
Memory efficiency: Very efficient for small hashes (< 512 fields)
Memory: ~40 bytes overhead per hash + field/value sizes
Best for: Objects, records, small collections of related data

Streams: Event Sourcing and Real-time Data

What are Redis Streams?

Redis Streams are append-only data structures that model a log of events. They're designed for real-time data processing, message queuing, and event sourcing patterns. Streams automatically generate unique IDs and support consumer groups for scalable processing.

Stream Commands

Adding Entries

# Add entry with auto-generated ID
XADD sensor:temperature * temp 23.5 humidity 65 timestamp 1671234567

# Add entry with specific ID
XADD sensor:temperature 1671234567000-0 temp 24.1 humidity 68

# Add with maxlen to limit stream size
XADD sensor:temperature MAXLEN ~ 1000 * temp 22.8 humidity 70

Reading Entries

# Read all entries
XRANGE sensor:temperature - +

# Read entries in time range
XRANGE sensor:temperature 1671234567000 1671234580000

# Read latest entries
XREVRANGE sensor:temperature + - COUNT 10

# Read from specific ID onward
XREAD STREAMS sensor:temperature 1671234567000-0

Blocking Reads

# Block until new entries arrive
XREAD BLOCK 5000 STREAMS sensor:temperature $

# Read from multiple streams
XREAD BLOCK 0 STREAMS sensor:temperature sensor:pressure $ $

Consumer Groups

# Create consumer group
XGROUP CREATE sensor:temperature processing $ MKSTREAM

# Read as part of consumer group
XREADGROUP GROUP processing consumer1 COUNT 1 STREAMS sensor:temperature >

# Acknowledge processed message
XACK sensor:temperature processing 1671234567000-0

# Get pending messages
XPENDING sensor:temperature processing

Common Use Cases for Streams

IoT Data Collection

# Collect sensor data
XADD sensors:device123 * temperature 25.3 humidity 60 battery 85

# Read recent sensor data
XREVRANGE sensors:device123 + - COUNT 100

# Process data with consumer groups
XGROUP CREATE sensors:device123 analytics $
XREADGROUP GROUP analytics worker1 STREAMS sensors:device123 >

Event Sourcing

# Record user events
XADD user:events * user_id 1000 event login ip 192.168.1.1
XADD user:events * user_id 1000 event view_product product_id 123
XADD user:events * user_id 1000 event purchase order_id 456

# Replay user events
XRANGE user:events - +

Chat and Messaging

# Add chat messages
XADD chat:room123 * user alice message "Hello everyone!"
XADD chat:room123 * user bob message "Hi Alice!"

# Get recent messages
XREVRANGE chat:room123 + - COUNT 50

# Real-time message consumption
XREAD BLOCK 0 STREAMS chat:room123 $

Performance Characteristics

Append operation: O(1)
Range queries: O(log N + M) where M is the number of entries returned
Memory: Efficient for time-series data
Best for: Event sourcing, real-time analytics, message queues

Geospatial: Location-based Data

What are Redis Geospatial Indexes?

Redis Geospatial indexes allow you to store and query location data efficiently. Built on top of Sorted Sets, they provide commands for storing coordinates and performing location-based searches like finding nearby points.

Geospatial Commands

Adding Locations

# Add locations (longitude, latitude, member)
GEOADD locations -74.006 40.7128 "New York"
GEOADD locations -118.2437 34.0522 "Los Angeles"
GEOADD locations -87.6298 41.8781 "Chicago"

# Add multiple locations at once
GEOADD stores -73.9857 40.7484 "store:manhattan" -73.9442 40.8081 "store:bronx"

Getting Location Information

# Get coordinates
GEOPOS locations "New York" "Chicago"

# Get distance between points
GEODIST locations "New York" "Los Angeles" km

# Get geohash
GEOHASH locations "New York"

Proximity Searches

# Find locations within radius
GEORADIUS locations -74.006 40.7128 100 km

# Find locations within radius of a member
GEORADIUSBYMEMBER locations "New York" 500 km

# Get additional information
GEORADIUS locations -74.006 40.7128 100 km WITHCOORD WITHDIST WITHGEOHASH

# Count locations in radius
GEORADIUS locations -74.006 40.7128 100 km COUNT 5

Common Use Cases for Geospatial

Store Locator

# Add store locations
GEOADD stores -73.9857 40.7484 "store:1" -73.9442 40.8081 "store:2"

# Find stores near user location
GEORADIUS stores -73.9857 40.7484 5 km WITHDIST

# Find stores near specific store
GEORADIUSBYMEMBER stores "store:1" 10 km

Ride Sharing

# Track driver locations
GEOADD drivers -73.9857 40.7484 "driver:123" -73.9442 40.8081 "driver:456"

# Find nearby drivers
GEORADIUS drivers -73.9857 40.7484 2 km WITHDIST COUNT 5

# Update driver location
GEOADD drivers -73.9900 40.7500 "driver:123"

Delivery Tracking

# Track delivery locations
GEOADD deliveries -73.9857 40.7484 "delivery:789"

# Check if delivery is in area
GEORADIUS deliveries -73.9857 40.7484 1 km

Performance Characteristics

Add location: O(log N)
Radius search: O(N+log M) where N is the number of elements in the radius, M is the total number of elements
Distance calculation: O(log N)
Best for: Location-based services, proximity searches, mapping applications

Choosing the Right Data Structure

Decision Matrix

Use Case	Best Data Structure	Alternative	Why
Simple caching	String	Hash (for objects)	Simplest, fastest access
User session	Hash	String (serialized)	Easy field updates
Task queue	List	Stream	FIFO/LIFO operations
Leaderboard	Sorted Set	-	Score-based ranking
Tags/Categories	Set	-	Unique membership
Event logging	Stream	List	Append-only, time-ordered
Location search	Geospatial	-	Built-in distance calculations
Counters	String	Hash field	Atomic increment operations
Recent items	List	Sorted Set	Maintain insertion order
Shopping cart	Hash	Set (for simple items)	Item quantities as fields

Performance Guidelines

Memory Efficiency

Best: Hashes (for small objects), Strings
Good: Sets, Lists
Consider: Sorted Sets, Streams (higher overhead)

Access Patterns

Single item access: Strings, Hash fields
Range access: Lists, Sorted Sets, Streams
Set operations: Sets
Ranked access: Sorted Sets

Scalability Considerations

Large collections: Consider partitioning across multiple keys
High write volume: Streams with consumer groups
Complex queries: Consider Redis modules or external indexing

Best Practices

Key Naming Conventions

# Use descriptive, hierarchical names
user:1000:profile
user:1000:sessions
product:123:details
cache:api:weather:london

# Include data type in name when helpful
queue:email_notifications
set:user:1000:followers
zset:game:leaderboard

Memory Optimization

Use Hashes for small objects (< 512 fields)
Set appropriate expiration times
Use compressed serialization for large values
Monitor memory usage with INFO memory

Performance Tips

Avoid KEYS command in production (use SCAN)
Use pipelining for multiple operations
Prefer batch operations (MSET, HMSET, etc.)
Monitor slow log for performance issues

Data Modeling Guidelines

Design for your access patterns
Denormalize data for performance
Use appropriate data structures for each use case
Consider using multiple structures for complex entities

Common Anti-patterns

Wrong Data Structure Choice

Using Lists for membership tests: Use Sets instead
Using Sets for ranking: Use Sorted Sets instead
Using Strings for structured data: Use Hashes instead

Inefficient Operations

Frequent LINDEX on large lists: Consider alternative structures
Not using batch operations: Use MGET, HMGET, etc.
Scanning entire structures: Use cursors and limits

Memory Issues

No expiration on temporary data: Set TTL appropriately
Very large single structures: Consider partitioning
Wrong encoding settings: Optimize hash and set configurations

Next Steps

Now that you understand Redis data structures in depth, you're ready to build sophisticated applications. In the next posts of this series, we'll explore:

Essential Redis Commands: Master daily operations and CLI usage
Persistence Strategies: Ensure data durability in production
Advanced Features: Transactions, Lua scripting, and pub/sub

Key Takeaways

Redis offers seven distinct data structures, each optimized for specific use cases
Choose data structures based on access patterns and performance requirements
Strings are versatile but don't overlook specialized structures
Hashes are memory-efficient for small objects and structured data
Lists excel at ordered data and queue operations
Sets provide unique collections and set operations
Sorted Sets enable ranking and time-based operations
Streams are perfect for event sourcing and real-time data
Geospatial indexes simplify location-based features
Proper key naming and memory management are crucial for production use

Practice Exercises

Design a social media feed using Lists and Sets
Implement a voting system using Sorted Sets
Create a session store using Hashes with TTL
Build a real-time chat using Streams
Design a location-based recommendation system using Geospatial indexes

Report Abuse

Redis Data Structures Deep Dive: Mastering the Building Blocks

Introduction

Overview of Redis Data Structures

Strings: The Foundation

What are Redis Strings?

String Commands

Basic Operations

Numeric Operations

String Manipulation

Common Use Cases for Strings

Caching

Counters and Analytics

Feature Flags

Performance Characteristics

Lists: Ordered Collections

What are Redis Lists?

List Commands

Adding Elements

Retrieving Elements

Removing Elements

Blocking Operations

Common Use Cases for Lists

Task Queues

Activity Feeds

Recently Viewed Items

Performance Characteristics

Sets: Unique Collections

What are Redis Sets?

Set Commands

Basic Operations

Random Operations

Set Operations

Common Use Cases for Sets

Tags and Categories

Social Features

Unique Visitors Tracking

Blacklists and Whitelists

Performance Characteristics

Sorted Sets: Ranked Collections

What are Redis Sorted Sets?

Sorted Set Commands

Adding and Updating

Retrieving by Rank

Retrieving by Score

Information and Removal

Common Use Cases for Sorted Sets

Gaming Leaderboards

Priority Queues

Time-based Data

Rate Limiting

Performance Characteristics

Hashes: Object-like Structures

What are Redis Hashes?

Hash Commands

Setting and Getting Fields

Field Operations

Numeric Operations

Common Use Cases for Hashes

User Profiles

Shopping Carts

Configuration Settings

Product Information

Performance Characteristics

Streams: Event Sourcing and Real-time Data

What are Redis Streams?

Stream Commands

Adding Entries

Reading Entries

Blocking Reads

Consumer Groups

Common Use Cases for Streams

IoT Data Collection

Event Sourcing

Chat and Messaging

Performance Characteristics

Geospatial: Location-based Data

What are Redis Geospatial Indexes?

Geospatial Commands

Adding Locations