List Comprehensions - The elegant way of creating lists in python

Python is a beautiful programming language.In the recent years, it has become one of the most popular languages and it’s usage in the software industry is rapidly growing.It is also used heavily in the field of machine learning and artificial intelligence.Being a general purpose language , python has a huge community of people from all walks of life who use it to do the most amazing things.

I have coded in other languages like C, C++ , java , R but none of them are as elegant as python. I mostly use python for data science and ML but there is nothing you cannot do with python. In this blog post we will explore an elegant and faster way of creating lists in python , list comprehensions.

What are list comprehensions?
List comprehensions provide a concise way to create lists .Common applications are to make new lists where each element is the result of some operations applied to each member of another sequence or iterable, or to create a subsequence of those elements that satisfy a certain condition. From python docs

Why should you use list comprehensions?

1.They are easy to read and understand
As mentioned before , list comprehensions provide a concise way of creating lists . But what does that actually mean?Let’s look at an example.

In the example given below , we create a list of even numbers from 1 to 20 using a for loop and then a list comprehension .This is a pretty simple task and does not require a lot of code but it shows us how concise and elegant list comprehensions are.

Implementation using a for loop

l=[]
for x in range(1,20):
  if x%2==0:
    l.append(x)
l

[2, 4, 6, 8, 10, 12, 14, 16, 18]

Implementation using a list comprehension

[x for x in range(1,20) if x%2==0]

[2, 4, 6, 8, 10, 12, 14, 16, 18]

Wow!!! We just needed to write one line of code to create a list that contains all the even numbers from 1 to 20 (20 not included) . Although , the above example performs a very simple task , we now know just how powerful list comprehensions are.
Even if we have to do something complex it is always better to use list comprehensions . Since all the logic is packed in one line of code , it is much easier to debug as compared to a for loop with lots of lines and lots of indentation.

2.They are generally faster than for loops

List comprehensions are gnerally faster than for loops. There are mainly two reasons :

Python is written in C. List comprehensions do a much better job of moving the computation to the C level as compared to the slower computation done by python while we are using a for loop.
While using the for loop , you have to use the append() function in each iteration to add an element to the list you want to create. Calling the append() function each time slows down the execution .

But why should you believe me? Test it out for yourself …
Python has an in-built module called time that provides us with easy to use functions to track the amount of time our program runs for.

Importing the time library

import time

In this example, we will create a list that contains squares of even numbers and cubes of odd numbers. We will also calculate the amount of time each method takes to complete the computation.

Implementation using for loop

start=time.time()
l=[]
for i in range(0,20):
  if i%2==0:
    l.append(i**2)
  else:
    l.append(i**3)
print(l)
print("The execution time is:",time.time()-start)

[0, 1, 4, 27, 16, 125, 36, 343, 64, 729, 100, 1331, 144, 2197, 196, 3375, 256, 4913, 324, 6859]
The execution time is: 0.010234832763671875

Implementation using a list comprehension

start=time.time()
l=[x**2 if x % 2 == 0 else x**3 for x in range(0,20)]
print(l)
print("The execution time is:",time.time()-start)

[0, 1, 4, 27, 16, 125, 36, 343, 64, 729, 100, 1331, 144, 2197, 196, 3375, 256, 4913, 324, 6859]
The execution time is: 0.00024056434631347656

Well as you can clearly see, list comprehensions are way faster . But maybe this is just dumb luck . Let’s find a better way to validate the speed of list comprehensions.

The best way to test our program is to use different values for our variables and run the program over and over . Earlier , we only calculated the squares and cubes of numbers in the range [1,20] . Now , we are going to use multiple ranges of numbers and see how our execution time changes . Then we wil compare the execution time of the for loop and the list comprehension.

Defining the ranges

ranges=[[0,20],[0,50],[0,100],[0,500],[0,1000]]

Creating a function that uses for loop to complete the given task

def for_loop(r):
  start=time.time()
  l=[]
  for i in range(r[0],r[1]):
    if i%2==0:
      l.append(i**2)
    else:
      l.append(i**3)
  print("The execution time is {} seconds".format(time.time()-start))
  return l

How fast is the for loop at completing the task

for r in ranges:
  for_loop(r)

The execution time is 2.0742416381835938e-05 seconds
The execution time is 3.4332275390625e-05 seconds
The execution time is 5.817413330078125e-05 seconds
The execution time is 0.0003447532653808594 seconds
The execution time is 0.0009100437164306641 seconds

Creating a function that uses list comprehension to complete the given task

def list_comprehension(r):
  start=time.time()
  l=[x**2 if x % 2 == 0 else x**3 for x in range(r[0],r[1])]
  print("The execution time is {} seconds".format(time.time()-start))
  return l

How fast is the list comprehension at completing the task

for r in ranges:
  list_comprehension(r)

The execution time is 1.52587890625e-05 seconds
The execution time is 3.337860107421875e-05 seconds
The execution time is 6.0558319091796875e-05 seconds
The execution time is 0.0002884864807128906 seconds
The execution time is 0.0005509853363037109 seconds

Comparing the execution times , we observe that list comprehensions are faster even when we increase the range of numbers .Hence proved , list comprehensions are faster than for loops.

You can mess around with the above programs by changing the ranges or the function bodies .It is always better to play around with the code and try to break it .

For the curious minds out there , here is another method you can use to validate whether list comps are faster than for loops

Defining the ranges

ranges=[[0,20],[0,50],[0,100],[0,500],[0,1000]]

Creating a function that uses for loop to complete the given task

def for_loop(r):
  start=time.time()
  l=[]
  for i in range(r[0],r[1]):
    if i%2==0:
      l.append(i**2)
    else:
      l.append(i**3)
  return l, time.time()-start

Creating a function that uses list comprehension to complete the given task

def list_comprehension(r):
  start=time.time()
  l=[x**2 if x % 2 == 0 else x**3 for x in range(r[0],r[1])]
  return l,time.time()-start

Let’s see who is faster

for r in ranges:
  _,t_for=for_loop(r)
  _,t_list=list_comprehension(r)
  if t_for>t_list:
    print("List comprehension was faster in case:",r)
  else:
    print("For loop was faster in case:",r)

List comprehension was faster in case: [0, 20]
List comprehension was faster in case: [0, 50]
List comprehension was faster in case: [0, 100]
List comprehension was faster in case: [0, 500]
List comprehension was faster in case: [0, 1000]

What else ?

It’s not just about readability and speed . List comprehensions are awesome .

Let’s see what else we can do with them.

1.Nested ifs
We can use multiple if conditions in list comps. This increases the flexibility we have in terms of selecting elements from a range or a pre-defined list.It also makes it easier to manage multiple conditions.

Implementation using a for loop

l=[]
for y in range(0,100):
  if y % 2 ==0:
    if y % 5 == 0: 
      if y % 3 == 0:
        l.append(y)
l

[0, 30, 60, 90]

Implementation using a list comprehension

[y for y in range(100) if y % 2 ==0 if y % 5 == 0 if y % 3==0 ]

[0, 30, 60, 90]

2.Working with strings
Often, we have to create lists from strings using some conditions or functions . Working with strings can become cumbersome if you are using a for loop , or worse , multiple for loops . Here , we will see how list comps can be used to find out the indices of all the vowels in a string.

The string

word="pneumonoultramicroscopicsilicovolcanoconiosis"

Creating a list of vowels

vowels=["a","e","i","o","u"]

Implementation using for loop

l=[]
for idx,x in enumerate(word):
  if x in vowels:
    l.append({x:idx})
l

[{'e': 2},
 {'u': 3},
 {'o': 5},
 {'o': 7},
 {'u': 8},
 {'a': 12},
 {'i': 14},
 {'o': 17},
 {'o': 20},
 {'i': 22},
 {'i': 25},
 {'i': 27},
 {'o': 29},
 {'o': 31},
 {'a': 34},
 {'o': 36},
 {'o': 38},
 {'i': 40},
 {'o': 41},
 {'i': 43}]

Implementation using list comprehension

[{x:idx} for idx,x in enumerate(word) if x in vowels]

[{'e': 2},
 {'u': 3},
 {'o': 5},
 {'o': 7},
 {'u': 8},
 {'a': 12},
 {'i': 14},
 {'o': 17},
 {'o': 20},
 {'i': 22},
 {'i': 25},
 {'i': 27},
 {'o': 29},
 {'o': 31},
 {'a': 34},
 {'o': 36},
 {'o': 38},
 {'i': 40},
 {'o': 41},
 {'i': 43}]

3.Multiple for loops
The world is a simulation and the source code has multiple for loops. The ability to use multiple for loops to perform complex tasks is very important in the programming world . List comps allow us to use as many for loops as we want inside the brackets.

l=[]
for i in [3,5,7,9]:
  for j in [2,4,6,8]:
    if i**j > j**i:
      l.append(i**j)
    else:
      l.append(j**i)
l

[i**j if i**j > j**i else j**i for i in [3,5,7,9] for j in [2,4,6,8]]

4.Nested list comprehensions
Often , we need to create a list that is a collection of multiple lists. Using nested list comps , we can again do this with one line of code . Suppose you have to create a program that prints multiplication tables, how would you do it ? Would you use for loops ? How many loops must you create? Also what form should the output be in?
Let’s see an easy way you could accomplish this task using list comps.

Implementation using for loop

l=[]
for x in range(11):
  l1=[]
  for y in range(11):
    l1.append(x*y)
  l.append(l1)
l

[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30],
 [0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50],
 [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
 [0, 7, 14, 21, 28, 35, 42, 49, 56, 63, 70],
 [0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80],
 [0, 9, 18, 27, 36, 45, 54, 63, 72, 81, 90],
 [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100]]

Implementation using list comprehension

[[x*y for y in range(11)] for x in range(11)]

[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
 [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20],
 [0, 3, 6, 9, 12, 15, 18, 21, 24, 27, 30],
 [0, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40],
 [0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50],
 [0, 6, 12, 18, 24, 30, 36, 42, 48, 54, 60],
 [0, 7, 14, 21, 28, 35, 42, 49, 56, 63, 70],
 [0, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80],
 [0, 9, 18, 27, 36, 45, 54, 63, 72, 81, 90],
 [0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100]]

Conclusion

List comps are very versatile and you can use them in your projects to make your code faster , readable and modular. But more than that , list comps represent a better way of coding . A way of coding which involves getting rid of excessive clutter , making your programs concise and easier for people to understand.Getting rid of excessive clutter does not only mean making your program smaller, it means formulating your problem in the best possible way.

Albert Einstein used to say: > If you can’t explain it to a six year old, you don’t understand it yourself.

If you can formulate your problem such that you can explain it with one line of code then you have a deep understanding about your problem . This will help you create better systems that not only perform complex tasks but are easier to understand in how they work .

##A challenge for you

Okay!!! This has been a fascinating journey . But every journey must end for a new journey to begin .
For your new journey to begin , here is a challenge. Given below is a program that can be used to create a list of numbers from the Fibonacci Series , your task is to implement this using list comprehension.
It is even possible ? If not , why ? Or maybe there is a way ?

See you on the next adventure.

n=int(input("Enter the number of digits you want:"))
a=0 #first seed
b=1 #second seed
l=[1]
for _ in range(0,n-1):
  sum=a+b 
  l.append(sum)
  a=b
  b=sum
l

Enter the number of digits you want:10

[1, 1, 2, 3, 5, 8, 13, 21, 34, 55]