Python List Comprehensions on Steroids: Write 5-10x Faster Code

List comprehensions are Python's secret weapon for data processing. They're not just syntactic sugar—they're genuinely faster than equivalent for loops, often by 2-5x. Combined with itertools, you can process millions of items elegantly and efficiently.

This is the difference between Python code that crawls and Python code that flies.

List Comprehension Basics

The Pattern

# Basic pattern
[expression for item in iterable]

# With condition
[expression for item in iterable if condition]

# With transformation
[transform(item) for item in iterable]

Loop vs Comprehension

# Slow: Traditional loop
squares = []
for x in range(1000):
    squares.append(x ** 2)

# Fast: List comprehension (2-5x faster)
squares = [x ** 2 for x in range(1000)]

Common Patterns

Filtering

# Get even numbers
evens = [x for x in range(100) if x % 2 == 0]

# Filter by condition
adults = [p for p in people if p.age >= 18]

# Multiple conditions
results = [x for x in data if x > 0 and x < 100]

Transforming

# Uppercase all strings
upper = [s.upper() for s in strings]

# Extract field from objects
names = [user.name for user in users]

# Apply function
processed = [process(item) for item in items]

Nested Loops (Flattening)

# Flatten 2D list
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [num for row in matrix for num in row]
# Result: [1, 2, 3, 4, 5, 6, 7, 8, 9]

# All combinations
pairs = [(x, y) for x in [1, 2, 3] for y in ['a', 'b', 'c']]

Dict and Set Comprehensions

Dictionary Comprehensions

# Create dict from lists
names = ['a', 'b', 'c']
values = [1, 2, 3]
d = {k: v for k, v in zip(names, values)}

# Transform dict
d = {k: v * 2 for k, v in original.items()}

# Filter dict
filtered = {k: v for k, v in d.items() if v > 10}

Set Comprehensions

# Unique values with transformation
unique_lengths = {len(word) for word in words}

# Unique filtered values
unique_even = {x for x in numbers if x % 2 == 0}

Itertools Power-Ups

For complex operations, itertools provides optimized building blocks:

from itertools import chain, combinations, permutations, groupby, islice

chain - Flatten Iterables

# Combine multiple iterables
from itertools import chain
combined = list(chain([1, 2], [3, 4], [5, 6]))
# [1, 2, 3, 4, 5, 6]

combinations and permutations

# All pairs (unordered)
from itertools import combinations
pairs = list(combinations([1, 2, 3], 2))
# [(1, 2), (1, 3), (2, 3)]

# All arrangements (ordered)
from itertools import permutations
perms = list(permutations([1, 2, 3], 2))
# [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]

groupby - Group Consecutive Items

from itertools import groupby
data = [('a', 1), ('a', 2), ('b', 3), ('b', 4)]
grouped = {k: list(v) for k, v in groupby(data, key=lambda x: x[0])}
# {'a': [('a', 1), ('a', 2)], 'b': [('b', 3), ('b', 4)]}

islice - Slice Any Iterator

from itertools import islice
# First 10 items of any iterator
first_ten = list(islice(huge_generator, 10))

Generator Expressions (Memory Efficient)

For large datasets, use generator expressions instead of list comprehensions:

# List comprehension: creates entire list in memory
squares = [x**2 for x in range(1000000)]  # Uses ~8MB

# Generator expression: yields one at a time
squares = (x**2 for x in range(1000000))  # Uses ~100 bytes

# Use generators with functions that accept iterables
sum_squares = sum(x**2 for x in range(1000000))  # Memory efficient

Real-World Examples

Data Cleaning

# Clean and filter CSV data
clean_rows = [
    {k: v.strip() for k, v in row.items()}
    for row in csv_data
    if row.get('email') and '@' in row['email']
]

API Response Processing

# Extract specific fields from API response
users = [
    {'id': u['id'], 'name': u['profile']['name']}
    for u in api_response['users']
    if u.get('active', False)
]

File Processing

# Read and process file lines
with open('data.txt') as f:
    processed = [line.strip().upper() for line in f if line.strip()]

Performance Comparison

import timeit

# Test: square numbers 0-9999
# Loop: ~1.2ms
# Comprehension: ~0.7ms
# map(): ~0.6ms

# For simple operations, all are fine
# For complex operations or large data, comprehensions shine

Conclusion

List comprehensions and itertools are core Python skills that separate beginners from proficient developers. They're not just about writing less code—they express intent more clearly and run faster.

Start by converting your simple for loops to comprehensions. Once comfortable, explore itertools for more complex data transformations. Your code will be shorter, faster, and more Pythonic.