Python itertools Module

The itertools module in Python provides a collection of fast, memory-efficient tools for performing iterators. These functions work on iterables and produce iterators as output. The module is a standard library, so no installation is required.

Table of Contents

1. Introduction
2. Infinite Iterators
   • count
   • cycle
   • repeat
3. Iterators Terminating on the Shortest Input Sequence
   • accumulate
   • chain
   • compress
   • dropwhile
   • filterfalse
   • groupby
   • islice
   • starmap
   • takewhile
   • tee
   • zip_longest
4. Combinatoric Iterators
   • product
   • permutations
   • combinations
   • combinations_with_replacement
5. Examples
   • Basic Usage of Infinite Iterators
   • Using chain to Flatten a List of Lists
   • Generating Permutations and Combinations
6. Real-World Use Case
7. Conclusion
8. References

Introduction

The itertools module provides a set of functions for creating and using iterators. These iterators can be used to perform various operations such as concatenation, grouping, filtering, and more.

Infinite Iterators

count

Returns an iterator that generates consecutive integers, starting from the provided start value and incremented by step.

import itertools

counter = itertools.count(start=10, step=2)
for _ in range(5):
    print(next(counter))

Output:

10
12
14
16
18

cycle

Returns an iterator that cycles through the elements in the provided iterable indefinitely.

import itertools

cycler = itertools.cycle('ABCD')
for _ in range(8):
    print(next(cycler))

Output:

A
B
C
D
A
B
C
D

repeat

Returns an iterator that repeats the provided value indefinitely or up to a specified number of times.

import itertools

repeater = itertools.repeat('Hello', 3)
for item in repeater:
    print(item)

Output:

Hello
Hello
Hello

Iterators Terminating on the Shortest Input Sequence

accumulate

Returns an iterator that generates accumulated sums, or accumulated results of other binary functions provided via the func argument.

import itertools

data = [1, 2, 3, 4, 5]
accumulated = itertools.accumulate(data)
print(list(accumulated))

Output:

[1, 3, 6, 10, 15]

chain

Returns an iterator that generates elements from the first iterable until it is exhausted, then from the next iterable, and so on.

import itertools

chain = itertools.chain([1, 2, 3], ['a', 'b', 'c'])
print(list(chain))

Output:

[1, 2, 3, 'a', 'b', 'c']

compress

Returns an iterator that filters elements from the first iterable, only returning those that have a corresponding element in selectors that evaluates to True.

import itertools

data = ['a', 'b', 'c', 'd']
selectors = [1, 0, 1, 0]
compressed = itertools.compress(data, selectors)
print(list(compressed))

Output:

['a', 'c']

dropwhile

Returns an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element.

import itertools

data = [1, 2, 3, 4, 5]
dropped = itertools.dropwhile(lambda x: x < 3, data)
print(list(dropped))

Output:

[3, 4, 5]

filterfalse

Returns an iterator that filters elements from the iterable, only returning those for which the predicate is false.

import itertools

data = [1, 2, 3, 4, 5]
filtered = itertools.filterfalse(lambda x: x % 2, data)
print(list(filtered))

Output:

[2, 4]

groupby

Returns an iterator that generates consecutive keys and groups from the iterable. The key is a function computing a key value for each element.

import itertools

data = sorted([('a', 1), ('b', 2), ('a', 3), ('b', 4)])
grouped = itertools.groupby(data, key=lambda x: x[0])
for key, group in grouped:
    print(key, list(group))

Output:

a [('a', 1), ('a', 3)]
b [('b', 2), ('b', 4)]

islice

Returns an iterator that generates selected elements from the iterable. If start is specified, it skips all preceding elements; step determines the interval between elements.

import itertools

data = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
sliced = itertools.islice(data, 2, 8, 2)
print(list(sliced))

Output:

[2, 4, 6]

starmap

Returns an iterator that computes the function using arguments obtained from the iterable.

import itertools
import operator

data = [(2, 3), (4, 5), (6, 7)]
starmapped = itertools.starmap(operator.mul, data)
print(list(starmapped))

Output:

[6, 20, 42]

takewhile

Returns an iterator that returns elements from the iterable as long as the predicate is true.

import itertools

data = [1, 2, 3, 4, 5]
taken = itertools.takewhile(lambda x: x < 4, data)
print(list(taken))

Output:

[1, 2, 3]

tee

Returns n independent iterators from a single iterable.

import itertools

data = [1, 2, 3, 4, 5]
iters = itertools.tee(data, 2)
for it in iters:
    print(list(it))

Output:

[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]

zip_longest

Returns an iterator that aggregates elements from each of the iterables. If the iterables are of uneven length, missing values are filled-in with fillvalue.

import itertools

data1 = [1, 2, 3]
data2 = ['a', 'b']
zipped = itertools.zip_longest(data1, data2, fillvalue='X')
print(list(zipped))

Output:

[(1, 'a'), (2, 'b'), (3, 'X')]

Combinatoric Iterators

product

Returns the Cartesian product of input iterables.

import itertools

prod = itertools.product([1, 2], ['a', 'b'])
print(list(prod))

Output:

[(1, 'a'), (1, 'b'), (2, 'a'), (2, 'b')]

permutations

Returns successive permutations of elements in the iterable.

import itertools

perm = itertools.permutations([1, 2, 3])
print(list(perm))

Output:

[(1, 2, 3), (1, 3, 2), (2, 1, 3), (2, 3, 1), (3, 1, 2), (3, 2, 1)]

combinations

Returns successive combinations of elements in the iterable.

import itertools

comb = itertools.combinations([1, 2, 3], 2)
print(list(comb))

Output:

[(1, 2), (1, 3), (2, 3)]

combinations_with_replacement

Returns successive combinations of elements in the iterable, allowing individual elements to be repeated more than once.

import itertools

comb_wr = itertools.combinations_with_replacement([1, 2, 3], 2)
print(list(comb_wr))

Output:

[(1, 1), (1, 2), (1, 3), (2, 2), (2, 3), (3, 3)]

Examples

Basic Usage of Infinite Iterators

import itertools

counter = itertools.count(start=5, step=5)
for _ in range(5):
    print(next(counter))

cycler = itertools.cycle('AB')
for _ in range(4):
    print(next(cycler))

repeater = itertools.repeat('Hello', 3)
for item in repeater:
    print(item)

Output:

5
10
15
20
25
A
B
A
B
Hello
Hello
Hello

Using chain to Flatten a List of Lists

import itertools

lists = [[1, 2, 3], [4, 5], [6, 7, 8]]
flattened = itertools.chain(*lists)
print(list(flattened))

Output:

[1, 2, 3, 4, 5, 6, 7, 8]

Generating Permutations and Combinations

import itertools

items = ['A', 'B', 'C']

perm = itertools.permutations(items)
print("Permutations:", list(perm))

comb = itertools.combinations(items, 2)
print("Combinations:", list(comb))

comb_wr = itertools.combinations_with_replacement(items, 2)
print("Combinations with Replacement:", list(comb_wr))

Output:

Permutations: [('A', 'B', 'C'), ('A', 'C', 'B'), ('B', 'A', 'C'), ('B', 'C', 'A'), ('C', 'A', 'B'), ('C', 'B', 'A')]
Combinations: [('A', 'B'), ('A', 'C'), ('B', 'C')]
Combinations with Replacement: [('A', 'A'), ('A', 'B'), ('A', 'C'), ('B', 'B'), ('B', 'C'), ('C', 'C')]

Real-World Use Case

Generating Passwords

Generate all possible passwords using a given set of characters and a fixed length.

import itertools

characters = 'abc123'
length = 3

passwords = itertools.product(characters, repeat=length)
for password in passwords:
    print(''.join(password))

Output:

aaa
aab
aac
aa1
aa2
aa3
aba
abb
abc
ab1
ab2
ab3
aca
acb
acc
ac1
ac2
ac3
a1a
a1b
a1c
a11
a12
a13
a2a
a2b
a2c
a21
a22
a23
a3a
a3b
a3c
a31
a32
a33
baa
bab
bac
ba1
ba2
ba3
bba
bbb
bbc
bb1
bb2
bb3
bca
bcb
bcc
bc1
bc2
bc3
b1a
b1b
b1c
b11
b12
b13
b2a
b2b
b2c
b21
b22
b23
b3a
b3b
b3c
b31
b32
b33
caa
cab
cac
ca1
ca2
ca3
cba
cbb
cbc
cb1
cb2
cb3
cca
ccb
ccc
cc1
cc2
cc3
c1a
c1b
c1c
c11
c12
c13
c2a
c2b
c2c
c21
c22
c23
c3a
c3b
c3c
c31
c32
c33
1aa
1ab
1ac
1a1
1a2
1a3
1ba
1bb
1bc
1b1
1b2
1b3
1ca
1cb
1cc
1c1
1c2
1c3
11a
11b
11c
111
112
113
12a
12b
12c
121
122
123
13a
13b
13c
131
132
133
2aa
2ab
2ac
2a1
2a2
2a3
2ba
2bb
2bc
2b1
2b2
2b3
2ca
2cb
2cc
2c1
2c2
2c3
21a
21b
21c
211
212
213
22a
22b
22c
221
222
223
23a
23b
23c
231
232
233
3aa
3ab
3ac
3a1
3a2
3a3
3ba
3bb
3bc
3b1
3b2
3b3
3ca
3cb
3cc
3c1
3c2
3c3
31a
31b
31c
311
312
313
32a
32b
32c
321
322
323
33a
33b
33c
331
332
333

Grouping Data

Group students by their grades.

import itertools

students = [('John', 'A'), ('Jane', 'B'), ('Dave', 'A'), ('Sue', 'B'), ('Tim', 'C')]
students.sort(key=lambda x: x[1])

grouped = itertools.groupby(students, key=lambda x: x[1])
for grade, group in grouped:
    print(f"Grade {grade}: {[student[0] for student in group]}")

Output:

Grade A: ['John', 'Dave']
Grade B: ['Jane', 'Sue']
Grade C: ['Tim']

Conclusion

The itertools module in Python provides a powerful set of functions for creating and manipulating iterators. These functions can be used to perform a variety of operations efficiently and with minimal memory usage.

References

• Python itertools module documentation