List Comprehensions in Python and other tricks

About Simeon Franklin

Software Dev for about 12 years
Desktop C/VB/C++/Delphi, then web Perl/PHP/Javascript/Python
recently became full-time Python instructor for Marakana.com
http://simeonfranklin.com / @simeonfranklin

List Processing

Many programming problems can be reduced to list processing

Show the total amount of sales <$10 for orders in Q4 of 2011
Calculate the pixel location of objects that should be on the screen
Highlight all the <li tags> that aren’t first in their <ul>

Think in terms of pipelines of data

All these problems can be described in terms of an input list that is filtered and then converted to an output list or value.

How do we write code to accomplish this task?

Iteratively

PHP

<?php
/* show total profit for orders < 10 in q4 2012 */
$records = array(array("total" => 8.50, "shipping" => 1.99, "year" => '2011', "month" => "Dec"),
                 array("total" => 8.00, "shipping" => 0, "year" => '2011', "month" => "Jun"),
                 array("total" => 1.00, "shipping" => 0, "year" => '2011', "month" => "Nov"),
                 array("total" => 85.00, "shipping" => 12.50, "year" => '2011', "month" => "Nov"),
                 array("total" => 17.00, "shipping" => 1.00, "year" => '2010', "month" => "Dec"));
$out = array();
foreach($records as $r){
  if(in_array($r['month'], array('Oct', 'Nov', 'Dec'))
     && $r['year'] == '2011'
     && $r['total'] < 10){
     $out[] = $r["total"] - $r['shipping'];
  }
}
var_dump($out);
$total = 0;
foreach($out as $profit){
   $total += $profit;
}
var_dump($total);
?>
array(2) {
  [0]=>
  float(6.51)
  [1]=>
  float(1)
}

float(7.51)

Can we do better?

If we can learn to think of pipelines of input and output data we can use functional programming techniques to solve these sorts of problems more elegantly.

<?php
function filter($r){
  if(in_array($r['month'], array('Oct', 'Nov', 'Dec'))
     && $r['year'] == '2011'
     && $r['total'] < 10){
     return True;
  }
  else{
    return False;
  }
}
function profit($r1){
    return $r1['total'] - $r1['shipping'];
}
function add($x, $y){
    return $x + $y;
}
$records = array(array("total" => 8.50, "shipping" => 1.99, "year" => '2011', "month" => "Dec"),
                 array("total" => 8.00, "shipping" => 0, "year" => '2011', "month" => "Jun"),
                 array("total" => 1.00, "shipping" => 0, "year" => '2011', "month" => "Nov"),
                 array("total" => 85.00, "shipping" => 12.50, "year" => '2011', "month" => "Nov"),
                 array("total" => 17.00, "shipping" => 1.00, "year" => '2010', "month" => "Dec"));
$out = array_filter($records, "filter");
$out = array_map('profit', $out); //arguments reversed. So PHP
$total = array_reduce($out, 'add');
var_dump($total);
?>
float(7.51)

Is this better?

array_map, array_filter, array_reduce

Logic in (reusable, testable, shareable) functions. +1
Less Code? +1
Less Understandable. -1

Python’s solution: List Comprehensions

Filtering/Mapping in one line

[(one item in output list) (for loop defining input list) (optional if statement)]
# or
[(map) (list) (optional filter)]

Example

>>> a_list = range(10)
>>> a_list
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> [i*2 for i in a_list] # Double the input numbers
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18]
>>> [i*2 for i in a_list if i % 2] # Double the odd numbers
[2, 6, 10, 14, 18]
>>> [i for i in a_list if i > 5] # greater than 5
[6, 7, 8, 9]

Benefits

No functions necessary for mnapping and filtering.
Readable (try reading it aloud)
Syntactic support for common pattern (ala foreach ⇒ for loop)

Example

Try it out with Python shell online: http://doc.pyschools.com/console

records = [{"total": 8.50, "shipping": 1.99, "year": '2011', "month": "Dec"},
           {"total": 8.00, "shipping": 0, "year": '2011', "month": "Jun"},
           {"total": 1.00, "shipping": 0, "year": '2011', "month": "Nov"},
           {"total": 85.00, "shipping": 12.50, "year": '2011', "month": "Nov"},
           {"total": 17.00, "shipping": 1.00, "year": '2010', "month": "Dec"}]

total = sum([r['total'] - r['shipping'] for r in records
                if r['month'] in ['Oct', 'Nov', 'Dec']
                   and r['year'] == '2011'
                   and r['total'] < 10])

← →