The os module

This module represents generic operating system functionality. This module is especially important if
you want to make your programs platform-independent i.e. it allows the program to be written such that
it will run on Linux as well as Windows without any problems and without requiring changes. An example
of this is using the os.sep variable instead of the operation system-specific path separator.
Some of the more useful parts of the os module are listed below Most of them are self-explanatory.

• The os.name string specifies which platform you are using, such as 'nt' for Windows and
'posix' for Linux/Unix users.
• The os.getcwd() function gets the current working directory i.e. the path of the directory from
which the curent Python script is working.
• The os.getenv() and os.putenv() functions are used to get and set environment variables
respectively.
• The os.listdir() function returns the name of all files and directories in the specified directory.
• The os.remove() function is used to delete a file.
• The os.system() function is used to run a shell command.
• The os.linesep string gives the line terminator used in the current platform. For example, Windows
uses '\r\n', Linux uses '\n' and Mac uses '\r'.
• The os.path.split() function returns the directory name and file name of the path.
>>> os.path.split('/home/swaroop/byte/code/poem.txt')
('/home/swaroop/byte/code', 'poem.txt')
• The os.path.isfile() and the os.path.isdir() functions check if the given path refers
to a file or directory respectively. Similarly, the os.path.exists() function is used to check if
a given path actually exists.

You can explore the Python Standard Documentation for more details on these functions and variables.
You can use help(sys), etc. as well.

The sys module

The sys module contains system-specific functionality. we have already seen that the sys.argv list
contains the command-line arguments.
Command Line Arguments

Example 14.1. Using sys.argv
#!/usr/bin/python
# Filename: cat.py
import sys
def readfile(filename):
'''Print a file to the standard output.'''
f = file(filename)
while True:
line = f.readline()
if len(line) == 0:
break
print line, # notice comma
f.close()
# Script starts from here
if len(sys.argv) < 2:
print 'No action specified.'
sys.exit()
if sys.argv[1].startswith('--'):
option = sys.argv[1][2:]
# fetch sys.argv[1] but without the first two characters
if option == 'version':
print 'Version 1.2'
elif option == 'help':
print '''\
This program prints files to the standard output.


Any number of files can be specified.
Options include:
--version : Prints the version number
--help : Display this help'''
else:
print 'Unknown option.'
sys.exit()
else:
for filename in sys.argv[1:]:
readfile(filename)


Output
$ python cat.py
No action specified.
$ python cat.py --help
This program prints files to the standard output.
Any number of files can be specified.
Options include:
--version : Prints the version number
--help : Display this help
$ python cat.py --version
Version 1.2
$ python cat.py --nonsense
Unknown option.
$ python cat.py poem.txt
Programming is fun
When the work is done
if you wanna make your work also fun:
use Python!


How It Works
This program tries to mimic the cat command familiar to Linux/Unix users. You just speicfy the names
of some text files and it will print them to the output.
When a Python program is run i.e. not an interactive mode, there is always at least one item in the
sys.argv list which is the name of the current program being run and is available as sys.argv[0]
since Python starts counting from 0. Other command line arguments follow this item.
To make the program user-friendly we have supplied certain options that the user can specify to learn
more about the program. We use the first argument to check if any options have been specified to our
program. If the --version option is used, the version number of the program is printed. Similarly,
when the --help option is specified, we give a bit of explanation about the program. We make use of
the sys.exit function to exit the running program. As always, see help(sys.exit) for more details.
When no options are specified and filenames are passed to the program, it simply prints out each line of
each file, one after the other in the order specified on the command line.
As an aside, the name cat is short for concatenate which is basically what this program does - it can
print out a file or attach/concatenate two or more files together in the output.

Try..Finally

What if you were reading a file and you wanted to close the file whether or not an exception was raised?
This can be done using the finally block. Note that you can use an except clause along with a finally
block for the same corresponding try block. You will have to embed one within another if you
want to use both.

Using Finally

Example 13.3. Using Finally
#!/usr/bin/python
# Filename: finally.py
import time
try:
f = file('poem.txt')
while True: # our usual file-reading idiom
line = f.readline()
if len(line) == 0:
break
time.sleep(2)
print line,
finally:
f.close()
print 'Cleaning up...closed the file'


Output
$ python finally.py
Programming is fun
When the work is done
Cleaning up...closed the file
Traceback (most recent call last):
File "finally.py", line 12, in ?
time.sleep(2)
KeyboardInterrupt


How It Works
We do the usual file-reading stuff, but I've arbitrarily introduced a way of sleeping for 2 seconds before
printing each line using the time.sleep method. The only reason is so that the program runs slowly
(Python is very fast by nature). When the program is still running, press Ctrl-c to interrupt/cancel the
program.
Observe that a KeyboardInterrupt exception is thrown and the program exits, but before the program
exits, the finally clause is executed and the file is closed.

Raising Exceptions

You can raise exceptions using the raise statement. You also have to specify the name of the error/
exception and the exception object that is to be thrown along with the exception. The error or exception
that you can arise should be class which directly or indirectly is a derived class of the Error or Exception
class respectively.

How To Raise Exceptions

Example 13.2. How to Raise Exceptions
#!/usr/bin/python
# Filename: raising.py
class ShortInputException(Exception):
'''A user-defined exception class.'''
def __init__(self, length, atleast):
Exception.__init__(self)
self.length = length
self.atleast = atleast
try:
s = raw_input('Enter something --> ')
if len(s) < 3:
raise ShortInputException(len(s), 3)
# Other work can continue as usual here
except EOFError:
print '\nWhy did you do an EOF on me?'
except ShortInputException, x:
print 'ShortInputException: The input was of length %d, \
was expecting at least %d' % (x.length, x.atleast)
else:
print 'No exception was raised.'

Output
$ python raising.py
Enter something -->
Why did you do an EOF on me?
$ python raising.py
Enter something --> ab
ShortInputException: The input was of length 2, was expecting at least 3
$ python raising.py
Enter something --> abc
No exception was raised.

How It Works

Here, we are creating our own exception type although we could've used any predefined exception/error
for demonstration purposes. This new exception type is the ShortInputException class. It has two
fields - length which is the length of the given input, and atleast which is the minimum length that
the program was expecting.
In the except clause, we mention the class of error as well as the variable to hold the corresponding error/
exception object. This is analogous to parameters and arguments in a function call. Within this particular
except clause, we use the length and atleast fields of the exception object to print an appropriate
message to the user.

Exceptions

Exceptions occur when certain exceptional situations occur in your program. For example, what if you
are going to read a file and the file does not exist? Or what if you accidentally deleted it when the program
was running? Such situations are handled using exceptions.
What if your program had some invalid statements? This is handled by Python which raises its hands
and tells you there is an error.

Errors

Consider a simple print statement. What if we misspelt print as Print? Note the capitalization. In
this case, Python raises a syntax error.
>>> Print 'Hello World'
File "<stdin>", line 1
Print 'Hello World'
^
SyntaxError: invalid syntax
>>> print 'Hello World'
Hello World


Observe that a SyntaxError is raised and also the location where the error was detected is printed.
This is what an error handler for this error does.

Try..Except

We will try to read input from the user. Press Ctrl-d and see what happens.
>>> s = raw_input('Enter something --> ')
Enter something --> Traceback (most recent call last):
File "<stdin>", line 1, in ?
EOFError


Python raises an error called EOFError which basically means it found an end of file when it did not
expect to (which is represented by Ctrl-d)
Next, we will see how to handle such errors.
Handling Exceptions
We can handle exceptions using the try..except statement. We basically put our usual statements
within the try-block and put all our error handlers in the except-block.

Example 13.1. Handling Exceptions
#!/usr/bin/python
# Filename: try_except.py
import sys
try:
s = raw_input('Enter something --> ')
except EOFError:
print '\nWhy did you do an EOF on me?'
sys.exit() # exit the program
except:
print '\nSome error/exception occurred.'
# here, we are not exiting the program
print 'Done'


Output
$ python try_except.py
Enter something -->
Why did you do an EOF on me?
$ python try_except.py
Enter something --> Python is exceptional!
Done


How It Works
We put all the statements that might raise an error in the try block and then handle all the errors and
exceptions in the except clause/block. The except clause can handle a single specified error or exception,
or a parenthesized list of errors/exceptions. If no names of errors or exceptions are supplied, it
will handle all errors and exceptions. There has to be at least one except clause associated with every
try clause.
If any error or exception is not handled, then the default Python handler is called which just stops the execution
of the program and prints a message. We have already seen this in action.
You can also have an else clause associated with a try..catch block. The else clause is executed
if no exception occurs.
We can also get the exception object so that we can retrieve additional information about the exception
which has occurred. This is demonstrated in the next example.

Pickle

Python provides a standard module called pickle using which you can store any Python object in a
file and then get it back later intact. This is called storing the object persistently.
There is another module called cPickle which functions exactly same as the pickle module except
that it is written in the C language and is (upto 1000 times) faster. You can use either of these modules,
although we will be using the cPickle module here. Remember though, that we refer to both these
modules as simply the pickle module.
Pickling and Unpickling

Example 12.2. Pickling and Unpickling
#!/usr/bin/python
# Filename: pickling.py
import cPickle as p
#import pickle as p
shoplistfile = 'shoplist.data' # the name of the file where we will store the object
shoplist = ['apple', 'mango', 'carrot']
# Write to the file
f = file(shoplistfile, 'w')
p.dump(shoplist, f) # dump the object to a file
f.close()
del shoplist # remove the shoplist
# Read back from the storage
f = file(shoplistfile)
storedlist = p.load(f)
print storedlist


Output
$ python pickling.py
['apple', 'mango', 'carrot']


How It Works
First, notice that we use the import..as syntax. This is handy since we can use a shorter name for a
module. In this case, it even allows us to switch to a different module (cPickle or pickle) by
simply changing one line! In the rest of the program, we simply refer to this module as p.
To store an object in a file, first we open a file object in write mode and store the object into the open
file by calling the dump function of the pickle module. This process is called pickling.
Next, we retrieve the object using the load function of the pickle module which returns the object.
This process is called unpickling.

Files

You can open and use files for reading or writing by creating an object of the file class and using its
read, readline or write methods appropriately to read from or write to the file. The ability to read
or write to the file depends on the mode you have specified for the file opening. Then finally, when you
are finished with the file, you call the close method to tell Python that we are done using the file.

Using file

Example 12.1. Using files
#!/usr/bin/python
# Filename: using_file.py
poem = '''\
Programming is fun
When the work is done
if you wanna make your work also fun:
use Python!
'''
f = file('poem.txt', 'w') # open for 'w'riting
f.write(poem) # write text to file
f.close() # close the file
f = file('poem.txt') # if no mode is specified, 'r'ead mode is assumed by default
while True:
line = f.readline()
if len(line) == 0: # Zero length indicates EOF
break
print line, # Notice comma to avoid automatic newline added by Python
f.close() # close the file


Output
$ python using_file.py
Programming is fun
When the work is done
if you wanna make your work also fun:
use Python!

How It Works
First, we create an instance of the file class by specifying the name of the file and the mode in which
we want to open the file. The mode can be a read mode ('r'), write mode ('w') or append mode
('a'). There are actually many more modes available and help(file) will give you more details
about them.
We first open the file in write mode and use the write method of the file class to write to the file
and then we finally close the file.
Next, we open the same file again for reading. If we don't specify a mode, then the read mode is the default
one. We read in each line of the file using the readline method, in a loop. This method returns a
complete line including the newline character at the end of the line. So, when an empty string is returned,
it indicates that the end of the file has been reached and we stop the loop.
Notice that we use a comma with the print statement to suppress the automatic newline that the
print statement adds because the line that is read from the file already ends with a newline character.
Then, we finally close the file.
Now, see the contents of the poem.txt file to confirm that the program has indeed worked properly.

Inheritance

One of the major benefits of object oriented programming is reuse of code and one of the ways this is
achieved is through the inheritance mechanism. Inheritance can be best imagined as implementing a
type and subtype relationship between classes.
Suppose you want to write a program which has to keep track of the teachers and students in a college.
They have some common characteristics such as name, age and address. They also have specific characteristics
such as salary, courses and leaves for teachers and, marks and fees for students.
You can create two independent classes for each type and process them but adding a new common characteristic
would mean adding to both of these independent classes. This quickly becomes unwieldy.
A better way would be to create a common class called SchoolMember and then have the teacher and
student classes inherit from this class i.e. they will become sub-types of this type (class) and then we can
add specific characteristics to these sub-types.
There are many advantages to this approach. If we add/change any functionality in SchoolMember,
this is automatically reflected in the subtypes as well. For example, you can add a new ID card field for
both teachers and students by simply adding it to the SchoolMember class. However, changes in the subtypes
do not affect other subtypes. Another advantage is that if you can refer to a teacher or student object
as a SchoolMember object which could be useful in some situations such as counting of the number
of school members. This is called polymorphism where a sub-type can be substituted in any situation
where a parent type is expected i.e. the object can be treated as an instance of the parent class.
Also observe that we reuse the code of the parent class and we do not need to repeat it in the different
classes as we would have had to in case we had used independent classes.
The SchoolMember class in this situation is known as the base class or the superclass. The Teacher
and Student classes are called the derived classes or subclasses.
We will now see this example as a program.

Using Inheritance

Example 11.5. Using Inheritance
#!/usr/bin/python
# Filename: inherit.py
class SchoolMember:
'''Represents any school member.'''
def __init__(self, name, age):
self.name = name
self.age = age
print '(Initialized SchoolMember: %s)' % self.name
def tell(self):
'''Tell my details.'''
print 'Name:"%s" Age:"%s"' % (self.name, self.age),
class Teacher(SchoolMember):
'''Represents a teacher.'''
def __init__(self, name, age, salary):
SchoolMember.__init__(self, name, age)
self.salary = salary
print '(Initialized Teacher: %s)' % self.name
def tell(self):
SchoolMember.tell(self)
print 'Salary: "%d"' % self.salary
class Student(SchoolMember):
'''Represents a student.'''
def __init__(self, name, age, marks):
SchoolMember.__init__(self, name, age)
self.marks = marks
print '(Initialized Student: %s)' % self.name
def tell(self):
SchoolMember.tell(self)
print 'Marks: "%d"' % self.marks
t = Teacher('Mrs. Shrividya', 40, 30000)
s = Student('Swaroop', 22, 75)
print # prints a blank line
members = [t, s]
for member in members:
member.tell() # works for both Teachers and Students


Output
$ python inherit.py
(Initialized SchoolMember: Mrs. Shrividya)
(Initialized Teacher: Mrs. Shrividya)
(Initialized SchoolMember: Swaroop)
(Initialized Student: Swaroop)
Name:"Mrs. Shrividya" Age:"40" Salary: "30000"
Name:"Swaroop" Age:"22" Marks: "75"

How It Works
To use inheritance, we specify the base class names in a tuple following the class name in the class
definition. Next, we observe that the __init__ method of the base class is explicitly called using the
self variable so that we can initialize the base class part of the object. This is very important to remember
- Python does not automatically call the constructor of the base class, you have to explicitly call
it yourself.
We also observe that we can call methods of the base class by prefixing the class name to the method
call and then pass in the self variable along with any arguments.
Notice that we can treat instances of Teacher or Student as just instances of the SchoolMember
when we use the tell method of the SchoolMember class.
Also, observe that the tell method of the subtype is called and not the tell method of the School-
Member class. One way to understand this is that Python always starts looking for methods in the type,
which in this case it does. If it could not find the method, it starts looking at the methods belonging to its
base classes one by one in the order they are specified in the tuple in the class definition.
A note on terminology - if more than one class is listed in the inheritance tuple, then it is called multiple
inheritance.

Class and Object Variables

We have already discussed the functionality part of classes and objects, now we'll see the data part of it.
Actually, they are nothing but ordinary variables which are bound to the classes and objects namespaces
i.e. the names are valid within the context of these classes and objects only.
There are two types of fields - class variables and object variables which are classified depending on
whether the class or the object owns the variables respectively.
Class variables are shared in the sense that they are accessed by all objects (instances) of that class.
There is only copy of the class variable and when any one object makes a change to a class variable, the change is reflected in all the other instances as well.

Object variables are owned by each individual object/instance of the class. In this case, each object has
its own copy of the field i.e. they are not shared and are not related in any way to the field by the samen
name in a different instance of the same class. An example will make this easy to understand.

Using Class and Object Variables

Example 11.4. Using Class and Object Variables
#!/usr/bin/python
# Filename: objvar.py
class Person:
'''Represents a person.'''
population = 0
def __init__(self, name):
'''Initializes the person's data.'''
self.name = name
print '(Initializing %s)' % self.name
# When this person is created, he/she
# adds to the population
Person.population += 1
def __del__(self):
'''I am dying.'''
print '%s says bye.' % self.name
Person.population -= 1
if Person.population == 0:
print 'I am the last one.'
else:
print 'There are still %d people left.' % Person.population
def sayHi(self):
'''Greeting by the person.
Really, that's all it does.'''
print 'Hi, my name is %s.' % self.name
def howMany(self):
'''Prints the current population.'''
if Person.population == 1:
print 'I am the only person here.'
else:
print 'We have %d persons here.' % Person.population
swaroop = Person('Swaroop')
swaroop.sayHi()
swaroop.howMany()
kalam = Person('Abdul Kalam')
kalam.sayHi()
kalam.howMany()
swaroop.sayHi()
swaroop.howMany()

Output
$ python objvar.py
(Initializing Swaroop)
Hi, my name is Swaroop.
I am the only person here.
(Initializing Abdul Kalam)
Hi, my name is Abdul Kalam.
We have 2 persons here.
Hi, my name is Swaroop.
We have 2 persons here.
Abdul Kalam says bye.
There are still 1 people left.
Swaroop says bye.
I am the last one.

How It Works
This is a long example but helps demonstrate the nature of class and object variables. Here, population
belongs to the Person class and hence is a class variable. The name variable belongs to the object
(it is assigned using self) and hence is an object variable.
Thus, we refer to the population class variable as Person.population and not as
self.population. Note that an object variable with the same name as a class variable will hide the
class variable! We refer to the object variable name using self.name notation in the methods of that
object. Remember this simple difference between class and object variables.
Observe that the __init__ method is used to initialize the Person instance with a name. In this
method, we increase the population count by 1 since we have one more person being added. Also
observe that the values of self.name is specific to each object which indicates the nature of object
variables.
Remember, that you must refer to the variables and methods of the same object using the self variable
only. This is called an attribute reference.
In this program, we also see the use of docstrings for classes as well as methods. We can access the
class docstring at runtime using Person.__doc__ and the method docstring as Person.
sayHi.__doc__
Just like the __init__ method, there is another special method __del__ which is called when an object
is going to die i.e. it is no longer being used and is being returned to the system for reusing that
piece of memory. In this method, we simply decrease the Person.population count by 1.
The __del__ method is run when the object is no longer in use and there is no guarantee when that
method will be run. If you want to explicitly do this, you just have to use the del statement which we
have used in previous examples.
Note for C++/Java/C# Programmers

All class members (including the data members) are public and all the methods are virtual in
Python.
One exception: If you use data members with names using the double underscore prefix such as
__privatevar, Python uses name-mangling to effectively make it a private variable.
Thus, the convention followed is that any variable that is to be used only within the class or object
should begin with an underscore and all other names are public and can be used by other
classes/objects. Remember that this is only a convention and is not enforced by Python (except
for the double underscore prefix).
Also, note that the __del__ method is analogous to the concept of a destructor.

The __init__ method

There are many method names which have special significance in Python classes. We will see the significance
of the __init__ method now.
The __init__ method is run as soon as an object of a class is instantiated. The method is useful to do
any initialization you want to do with your object. Notice the double underscore both in the beginning
and at the end in the name.

Using the __init__ method

Example 11.3. Using the __init__ method
#!/usr/bin/python
# Filename: class_init.py
class Person:
def __init__(self, name):
self.name = name
def sayHi(self):
print 'Hello, my name is', self.name
p = Person('Swaroop')
p.sayHi()
# This short example can also be written as Person('Swaroop').sayHi()


Output
$ python class_init.py
Hello, my name is Swaroop


How It Works
Here, we define the __init__ method as taking a parameter name (along with the usual self). Here,
we just create a new field also called name. Notice these are two different variables even though they
have the same name. The dotted notation allows us to differentiate between them.
Most importantly, notice that we do not explicitly call the __init__ method but pass the arguments in
the parentheses following the class name when creating a new instance of the class. This is the special
significance of this method.
Now, we are able to use the self.name field in our methods which is demonstrated in the sayHi
method.
Note for C++/Java/C# Programmers
The __init__ method is analogous to a constructor in C++, C# or Java.

object Methods

We have already discussed that classes/objects can have methods just like functions except that we have
an extra self variable. We will now see an example.

Using Object Methds

Example 11.2. Using Object Methods
#!/usr/bin/python
# Filename: method.py
class Person:
def sayHi(self):
print 'Hello, how are you?'
p = Person()
p.sayHi()
# This short example can also be written as Person().sayHi()

Output
$ python method.py
Hello, how are you?


How It Works
Here we see the self in action. Notice that the sayHi method takes no parameters but still has the
self in the function definition.

Classes

The simplest class possible is shown in the following example.

Creating a Class

Example 11.1. Creating a Class
#!/usr/bin/python
# Filename: simplestclass.py
class Person:
pass # An empty block
p = Person()
print p


Output
$ python simplestclass.py
<__main__.Person instance at 0xf6fcb18c>

How It Works
We create a new class using the class statement followed by the name of the class. This follows an indented
block of statements which form the body of the class. In this case, we have an empty block which
is indicated using the pass statement.
Next, we create an object/instance of this class using the name of the class followed by a pair of parentheses.
(We will learn more about instantiation in the next section). For our verification, we confirm the
type of the variable by simply printing it. It tells us that we have an instance of the Person class in the
__main__ module.
Notice that the address of the computer memory where your object is stored is also printed. The address
will have a different value on your computer since Python can store the object wherever it finds space.

Object-Oriented Programming Introduction

In all our programs till now, we have designed our program around functions or blocks of statements
which manipulate data. This is called the procedure-oriented way of programming. There is another way
of organizing your program which is to combine data and functionality and wrap it inside what is called
an object. This is called the object oriented programming paradigm. Most of the time you can use procedural
programming but sometimes when you want to write large programs or have a solution that is
better suited to it, you can use object oriented programming techniques.
Classes and objects are the two main aspecs of object oriented programming. A class creates a new type
where objects are instances of the class. An analogy is that you can have variables of type int which
translates to saying that variables that store integers are variables which are instances (objects) of the
int class.
Note for C/C++/Java/C# Programmers
Note that even integers are treated as objects (of the int class). This is unlike C++ and Java
(before version 1.5) where integers are primitive native types. See help(int) for more details
on the class.
C# and Java 1.5 programmers will be familiar with this concept since it is similar to the boxing
and unboxing concept.
Objects can store data using ordinary variables that belong to the object. Variables that belong to an object
or class are called as fields. Objects can also have functionality by using functions that belong to a
class. Such functions are called methods of the class. This terminology is important because it helps us
to differentiate between functions and variables which are separate by itself and those which belong to a
class or object. Collectively, the fields and methods can be referred to as the attributes of that class.
Fields are of two types - they can belong to each instance/object of the class or they can belong to the
class itself. They are called instance variables and class variables respectively.
A class is created using the class keyword. The fields and methods of the class are listed in an indented
block.
The self
Class methods have only one specific difference from ordinary functions - they must have an extra first
name that has to be added to the beginning of the parameter list, but you do do not give a value for this
parameter when you call the method, Python will provide it. This particular variable refers to the object
itself, and by convention, it is given the name self.
Although, you can give any name for this parameter, it is strongly recommended that you use the name
self - any other name is definitely frowned upon. There are many advantages to using a standard name
- any reader of your program will immediately recognize it and even specialized IDEs (Integrated Development
Environments) can help you if you use self.
Note for C++/Java/C# Programmers
The self in Python is equivalent to the self pointer in C++ and the this reference in Java and C#.
You must be wondering how Python gives the value for self and why you don't need to give a value
for it. An example will make this clear. Say you have a class called MyClass and an instance of this
class called MyObject. When you call a method of this object as MyObject.method(arg1,
arg2), this is automatically converted by Python into MyClass.method(MyObject, arg1,
arg2 - this is what the special self is all about.
This also means that if you have a method which takes no arguments, then you still have to define the
method to have a self argument.

The Software Development Process

We have now gone through the various phases in the process of writing a software. These phases can be
summarised as follows:
1. What (Analysis)
2. How (Design)
3. Do It (Implementation)
4. Test (Testing and Debugging)
5. Use (Operation or Deployment)
6. Maintain (Refinement)
Important
A recommended way of writing programs is the procedure we have followed in creating the
backup script - Do the analysis and design. Start implementing with a simple version. Test and
debug it. Use it to ensure that it works as expected. Now, add any features that you want and
continue to repeat the Do It-Test-Use cycle as many times as required. Remember, 'Software is
grown, not built'.

More Refinements

The fourth version is a satisfactorily working script for most users, but there is always room for improvement.
For example, you can include a verbosity level for the program where you can specify a -v
option to make your program become more talkative.
Another possible enhancement would be to allow extra files and directories to be passed to the script at
the command line. We will get these from the sys.argv list and we can add them to our source list
using the extend method provided by the list class.
One refinement I prefer is the use of the tar command instead of the zip command. One advantage is
that when you use the tar command along with gzip, the backup is much faster and the backup created
is also much smaller. If I need to use this archive in Windows, then WinZip handles such .tar.gz
files easily as well. The tar command is available by default on most Linux/Unix systems. Windows
users can download [http://gnuwin32.sourceforge.net/packages/tar.htm] and install it as well.
The command string will now be:
tar = 'tar -cvzf %s %s -X /home/swaroop/excludes.txt' % (target, ' '.join(srcdir))

The options are explained below.
• -c indicates creation of an archive.
• -v indicates verbose i.e. the command should be more talkative.
• -z indicates the gzip filter should be used.
• -f indicates force in creation of archive i.e. it should replace if there is a file by the same name
already.
• -X indicates a file which contains a list of filenames which must be excluded from the backup. For
example, you can specify *~ in this file to not include any filenames ending with ~ in the backup.
Important
The most preferred way of creating such kind of archives would be using the zipfile or
tarfile module respectively. They are part of the Python Standard Library and available for
you to use already. Using these libraries also avoids the use of the os.system which is generally
not advisable to use because it is very easy to make costly mistakes using it.
However, I have been using the os.system way of creating a backup purely for pedagogical
purposes, so that the example is simple enough to be understood by everybody but real enough
to be useful.

Problem Solving - Writing a Python Script

We have explored various parts of the Python language and now we will take a look at how all these
parts fit together, by designing and writing a program which does something useful.
The Problem
The problem is 'I want a program which creates a backup of all my important files'.
Although, this is a simple problem, there is not enough information for us to get started with the solution.
A little more analysis is required. For example, how do we specify which files are to be backed up?
Where is the backup stored? How are they stored in the backup?
After analyzing the problem properly, we design our program. We make a list of things about how our
program should work. In this case, I have created the following list on how I want it to work. If you do
the design, you may not come up with the same kind of problem - every person has their own way of doing
things, this is ok.
1. The files and directories to be backed up are specified in a list.
2. The backup must be stored in a main backup directory.
3. The files are backed up into a zip file.
4. The name of the zip archive is the current date and time.
5. We use the standard zip command available by default in any standard Linux/Unix distribution.
Windows users can use the Info-Zip program. Note that you can use any archiving command you
want as long as it has a command line interface so that we can pass arguments to it from our script.
The Solution
As the design of our program is now stable, we can write the code which is an implementation of our
solution.
First Version
Example 10.1. Backup Script - The First Version
#!/usr/bin/python
# Filename: backup_ver1.py
import os
import time
# 1. The files and directories to be backed up are specified in a list.
source = ['/home/swaroop/byte', '/home/swaroop/bin']

# If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] or something # 2. The backup must be stored in a main backup directory
target_dir = '/mnt/e/backup/' # Remember to change this to what you will be using
# 3. The files are backed up into a zip file.
# 4. The name of the zip archive is the current date and time
target = target_dir + time.strftime('%Y%m%d%H%M%S') + '.zip'
# 5. We use the zip command (in Unix/Linux) to put the files in a zip archive
zip_command = "zip -qr '%s' %s" % (target, ' '.join(source))
# Run the backup
if os.system(zip_command) == 0:
print 'Successful backup to', target
else:
print 'Backup FAILED'
Output
$ python backup_ver1.py
Successful backup to /mnt/e/backup/20041208073244.zip
Now, we are in the testing phase where we test that our program works properly. If it doesn't behave as
expected, then we have to debug our program i.e. remove the bugs (errors) from the program.
How It Works
You will notice how we have converted our design into code in a step-by-step manner.
We make use of the os and time modules and so we import them. Then, we specify the files and directories
to be backed up in the source list. The target directory is where store all the backup files and
this is specified in the target_dir variable. The name of the zip archive that we are going to create is
the current date and time which we fetch using the time.strftime() function. It will also have the
.zip extension and will be stored in the target_dir directory.
The time.strftime() function takes a specification such as the one we have used in the above program.
The %Y specification will be replaced by the year without the cetury. The %m specification will be
replaced by the month as a decimal number between 01 and 12 and so on. The complete list of such
specifications can be found in the [Python Reference Manual] that comes with your Python distribution.
Notice that this is similar to (but not same as) the specification used in print statement (using the %
followed by tuple).
We create the name of the target zip file using the addition operator which concatenates the strings i.e. it
joins the two strings together and returns a new one. Then, we create a string zip_command which
contains the command that we are going to execute. You can check if this command works by running it
on the shell (Linux terminal or DOS prompt).
The zip command that we are using has some options and parameters passed. The -q option is used to
indicate that the zip command should work quietly. The -r option specifies that the zip command
should work recursively for directories i.e. it should include subdirectories and files within the subdirectories as well. The two options are combined and specified in a shorter way as -qr. The options are
followed by the name of the zip archive to create followed by the list of files and directories to backup.
We convert the source list into a string using the join method of strings which we have already seen
how to use.
Then, we finally run the command using the os.system function which runs the command as if it was
run from the system i.e. in the shell - it returns 0 if the command was successfully, else it returns an error
number.
Depending on the outcome of the command, we print the appropriate message that the backup has failed
or succeeded and that's it, we have created a script to take a backup of our important files!
Note to Windows Users
You can set the source list and target directory to any file and directory names but you
have to be a little careful in Windows. The problem is that Windows uses the backslash (\) as
the directory separator character but Python uses backslashes to represent escape sequences!
So, you have to represent a backslash itself using an escape sequence or you have to use raw
strings. For example, use 'C:\\Documents' or r'C:\Documents' but do not use
'C:\Documents' - you are using an unknown escape sequence \D !
Now that we have a working backup script, we can use it whenever we want to take a backup of the
files. Linux/Unix users are advised to use the executable method as discussed earlier so that they can run
the backup script anytime anywhere. This is called the operation phase or the deployment phase of the
software.
The above program works properly, but (usually) first programs do not work exactly as you expect. For
example, there might be problems if you have not designed the program properly or if you have made a
mistake in typing the code, etc. Appropriately, you will have to go back to the design phase or you will
have to debug your program.
Second Version
The first version of our script works. However, we can make some refinements to it so that it can work
better on a daily basis. This is called the maintenance phase of the software.
One of the refinements I felt was useful is a better file-naming mechanism - using the time as the name
of the file within a directory with the current date as a directory within the main backup directory. One
advantage is that your backups are stored in a hierarchical manner and therefore it is much easier to
manage. Another advantage is that the length of the filenames are much shorter this way. Yet another
advantage is that separate directories will help you to easily check if you have taken a backup for each
day since the directory would be created only if you have taken a backup for that day.
Example 10.2. Backup Script - The Second Version
#!/usr/bin/python
# Filename: backup_ver2.py
import os
import time
# 1. The files and directories to be backed up are specified in a list.
source = ['/home/swaroop/byte', '/home/swaroop/bin']
# If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] or something

# 2. The backup must be stored in a main backup directory
target_dir = '/mnt/e/backup/' # Remember to change this to what you will be using
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
os.mkdir(today) # make directory
print 'Successfully created directory', today
# The name of the zip file
target = today + os.sep + now + '.zip'
# 5. We use the zip command (in Unix/Linux) to put the files in a zip archive
zip_command = "zip -qr '%s' %s" % (target, ' '.join(source))
# Run the backup
if os.system(zip_command) == 0:
print 'Successful backup to', target
else:
print 'Backup FAILED'
Output
$ python backup_ver2.py
Successfully created directory /mnt/e/backup/20041208
Successful backup to /mnt/e/backup/20041208/080020.zip
$ python backup_ver2.py
Successful backup to /mnt/e/backup/20041208/080428.zip
How It Works
Most of the program remains the same. The changes is that we check if there is a directory with the current
day as name inside the main backup directory using the os.exists function. If it doesn't exist,
we create it using the os.mkdir function.
Notice the use of os.sep variable - this gives the directory separator according to your operating system
i.e. it will be '/' in Linux, Unix, it will be '\\' in Windows and ':' in Mac OS. Using os.sep
instead of these characters directly will make our program portable and work across these systems.
Third Version
The second version works fine when I do many backups, but when there are lots of backups, I am finding
it hard to differentiate what the backups were for! For example, I might have made some major
changes to a program or presentation, then I want to associate what those changes are with the name of

the zip archive. This can be easily achieved by attaching a user-supplied comment to the name of the zip
archive.
Example 10.3. Backup Script - The Third Version (does not work!)
#!/usr/bin/python
# Filename: backup_ver2.py
import os
import time
# 1. The files and directories to be backed up are specified in a list.
source = ['/home/swaroop/byte', '/home/swaroop/bin']
# If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] or something # 2. The backup must be stored in a main backup directory
target_dir = '/mnt/e/backup/' # Remember to change this to what you will be using
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
# Take a comment from the user to create the name of the zip file
comment = raw_input('Enter a comment --> ')
if len(comment) == 0: # check if a comment was entered
target = today + os.sep + now + '.zip'
else:
target = today + os.sep + now + '_' +
comment.replace(' ', '_') + '.zip'
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
os.mkdir(today) # make directory
print 'Successfully created directory', today
# 5. We use the zip command (in Unix/Linux) to put the files in a zip archive
zip_command = "zip -qr '%s' %s" % (target, ' '.join(source))
# Run the backup
if os.system(zip_command) == 0:
print 'Successful backup to', target
else:
print 'Backup FAILED'
Output
$ python backup_ver3.py
File "backup_ver3.py", line 25
target = today + os.sep + now + '_' +
^
SyntaxError: invalid syntax


How This (does not) Work
This program does not work!. Python says there is a syntax error which means that the script does not
satisfy the structure that Python expects to see. When we observe the error given by Python, it also tells
us the place where it detected the error as well. So we start debugging our program from that line.
On careful observation, we see that the single logical line has been split into two physical lines but we
have not specified that these two physical lines belong together. Basically, Python has found the addition
operator (+) without any operand in that logical line and hence it doesn't know how to continue. Remember
that we can specify that the logical line continues in the next physical line by the use of a backslash
at the end of the physical line. So, we make this correction to our program. This is called bug fixing.
Fourth Version
Example 10.4. Backup Script - The Fourth Version
#!/usr/bin/python
# Filename: backup_ver2.py
import os, time
# 1. The files and directories to be backed up are specified in a list.
source = ['/home/swaroop/byte', '/home/swaroop/bin']
# If you are using Windows, use source = [r'C:\Documents', r'D:\Work'] or something # 2. The backup must be stored in a main backup directory
target_dir = '/mnt/e/backup/' # Remember to change this to what you will be using
# 3. The files are backed up into a zip file.
# 4. The current day is the name of the subdirectory in the main directory
today = target_dir + time.strftime('%Y%m%d')
# The current time is the name of the zip archive
now = time.strftime('%H%M%S')
# Take a comment from the user to create the name of the zip file
comment = raw_input('Enter a comment --> ')
if len(comment) == 0: # check if a comment was entered
target = today + os.sep + now + '.zip'
else:
target = today + os.sep + now + '_' + \
comment.replace(' ', '_') + '.zip'
# Notice the backslash!
# Create the subdirectory if it isn't already there
if not os.path.exists(today):
os.mkdir(today) # make directory
print 'Successfully created directory', today
# 5. We use the zip command (in Unix/Linux) to put the files in a zip archive
zip_command = "zip -qr '%s' %s" % (target, ' '.join(source))
# Run the backup

if os.system(zip_command) == 0:
print 'Successful backup to', target
else:
print 'Backup FAILED'


Output
$ python backup_ver4.py
Enter a comment --> added new examples
Successful backup to /mnt/e/backup/20041208/082156_added_new_examples.zip
$ python backup_ver4.py
Enter a comment -->
Successful backup to /mnt/e/backup/20041208/082316.zip
How It Works
This program now works! Let us go through the actual enhancements that we had made in version 3. We
take in the user's comments using the raw_input function and then check if the user actually entered
something by finding out the length of the input using the len function. If the user has just pressed
enter for some reason (maybe it was just a routine backup or no special changes were made), then we
proceed as we have done before.
However, if a comment was supplied, then this is attached to the name of the zip archive just before the
.zip extension. Notice that we are replacing spaces in the comment with underscores - this is because
managing such filenames are much easier.

More about Strings

We have already discussed strings in detail earlier. What more can there be to know? Well, did you
know that strings are also objects and have methods which do everything from checking part of a string
to stripping spaces!
The strings that you use in program are all objects of the class str. Some useful methods of this class
are demonstrated in the next example. For a complete list of such methods, see help(str).
String Methods
Example 9.7. String Methods
#!/usr/bin/python
# Filename: str_methods.py
name = 'Swaroop' # This is a string object
if name.startswith('Swa'):
print 'Yes, the string starts with "Swa"'
if 'a' in name:
print 'Yes, it contains the string "a"'
if name.find('war') != -1:
print 'Yes, it contains the string "war"'
delimiter = '_*_'
mylist = ['Brazil', 'Russia', 'India', 'China']
print delimiter.join(mylist)
Output
$ python str_methods.py
Yes, the string starts with "Swa"
Yes, it contains the string "a"
Yes, it contains the string "war"
Brazil_*_Russia_*_India_*_China
How It Works
Here, we see a lot of the string methods in action. The startswith method is used to find out whether
the string starts with the given string. The in operator is used to check if a given string is a part of the
string.
The find method is used to do find the position of the given string in the string or returns -1 if it is not
successful to find the substring. The str class also has a neat method to join the items of a sequence with the string acting as a delimiter between each item of the sequence and returns a bigger string generated
from this.

Objects and References

Example 9.6. Objects and References
#!/usr/bin/python
# Filename: reference.py
print 'Simple Assignment'
shoplist = ['apple', 'mango', 'carrot', 'banana']
mylist = shoplist # mylist is just another name pointing to the same object!
del shoplist[0] # I purchased the first item, so I remove it from the list
print 'shoplist is', shoplist
print 'mylist is', mylist
# notice that both shoplist and mylist both print the same list without
# the 'apple' confirming that they point to the same object
print 'Copy by making a full slice'
mylist = shoplist[:] # make a copy by doing a full slice
del mylist[0] # remove first item
print 'shoplist is', shoplist
print 'mylist is', mylist
# notice that now the two lists are different
Output
$ python reference.py
Simple Assignment
shoplist is ['mango', 'carrot', 'banana']
mylist is ['mango', 'carrot', 'banana']
Copy by making a full slice
shoplist is ['mango', 'carrot', 'banana']
mylist is ['carrot', 'banana']
How It Works
Most of the explanation is available in the comments itself. What you need to remember is that if you
want to make a copy of a list or such kinds of sequences or complex objects (not simple objects such as
integers), then you have to use the slicing operation to make a copy. If you just assign the variable name
to another name, both of them will refer to the same object and this could lead to all sorts of trouble if
you are not careful.
Note for Perl programmers
Remember that an assignment statement for lists does not create a copy. You have to use slicing
operation to make a copy of the sequence.

Sequences

Lists, tuples and strings are examples of sequences, but what are sequences and what is so special about
them? Two of the main features of a sequence is the indexing operation which allows us to fetch a particular
item in the sequence directly and the slicing operation which allows us to retrieve a slice of the
sequence i.e. a part of the sequence.
Using Sequences
Example 9.5. Using Sequences
#!/usr/bin/python
# Filename: seq.py
shoplist = ['apple', 'mango', 'carrot', 'banana']
# Indexing or 'Subscription' operation
print 'Item 0 is', shoplist[0]
print 'Item 1 is', shoplist[1]
print 'Item 2 is', shoplist[2]
print 'Item 3 is', shoplist[3]
print 'Item -1 is', shoplist[-1]
print 'Item -2 is', shoplist[-2]
# Slicing on a list
print 'Item 1 to 3 is', shoplist[1:3]
print 'Item 2 to end is', shoplist[2:]
print 'Item 1 to -1 is', shoplist[1:-1]
print 'Item start to end is', shoplist[:]
# Slicing on a string
name = 'swaroop'
print 'characters 1 to 3 is', name[1:3]
print 'characters 2 to end is', name[2:]
print 'characters 1 to -1 is', name[1:-1]
print 'characters start to end is', name[:]
Output
$ python seq.py
Item 0 is apple
Item 1 is mango
Item 2 is carrot
Item 3 is banana

Item -1 is banana
Item -2 is carrot
Item 1 to 3 is ['mango', 'carrot']
Item 2 to end is ['carrot', 'banana']
Item 1 to -1 is ['mango', 'carrot']
Item start to end is ['apple', 'mango', 'carrot', 'banana']
characters 1 to 3 is wa
characters 2 to end is aroop
characters 1 to -1 is waroo
characters start to end is swaroop
How It Works
First, we see how to use indexes to get individual items of a sequence. This is also referred to as the subscription
operation. Whenever you specify a number to a sequence within square brackets as shown
above, Python will fetch you the item corresponding to that position in the sequence. Remember that Python
starts counting numbers from 0. Hence, shoplist[0] fetches the first item and shoplist[3]
fetches the fourth item in the shoplist sequence.
The index can also be a negative number, in which case, the position is calculated from the end of the
sequence. Therefore, shoplist[-1] refers to the last item in the sequence and shoplist[-2]
fetches the second last item in the sequence.
The slicing operation is used by specifying the name of the sequence followed by an optional pair of
numbers separated by a colon within square brackets. Note that this is very very similar to the indexing
operation you have been using til lnow. Remember the numbers are optional but the colon isn't.
The first number (before the colon) in the slicing operation refers to the position from where the slice
starts and the second number (after the colon) indicates where the slice will stop at. If the first number is
not specified, Python will start at the beginning of the sequence. If the second number is left out, Python
will stop at the end of the sequence. Note that the slice returned starts at the start position and will end
just before the end position i.e. the start position is included but the end position is excluded from the sequence
slice.
Thus, shoplist[1:3] returns a slice of the sequence starting at position 1, includes position 2 but
stops at position 3 and therefore a slice of two items is returned. Similarly, shoplist[:] returns a
copy of the whole sequence.
You can also do slicing with negative positions. Negative numbers are used for positions from the end of
the sequence. For example, shoplist[:-1] will return a slice of the sequence which excludes the
last item of the sequence but contains everything else.
Try various combinations of such slice specifications using the Python interpreter interactively i.e. the
prompt so that you can see the results immediately. The great thing about sequences is that you can access
tuples, lists and strings all in the same way!

Dictionary

A dictionary is like an address-book where you can find the address or contact details of a person by
knowing only his/her name i.e. we associate keys (name) with values (details). Note that the key must
be unique just like you cannot find out the correct information if you have two persons with the exact
same name.
Note that you can use only immutable objects (like strings) for the keys of a dictionary but you can use
either immutable or mutable objects for the values of the dictionary. This basically translates to say that
you should use only simple objects for keys.
Pairs of keys and valus are specified in a dictionary by using the notation d = {key1 : value1,
key2 : value2 }. Notice that they key/value pairs are separated by a colon and the pairs are separated
themselves by commas and all this is enclosed in a pair of curly brackets.
Remember that key/value pairs in a dictionary are not ordered in any manner. If you want a particular
order, then you will have to sort them yourself before using it.
The dictionaries that you will be using are instances/objects of the dict class.
Using Dictionaries
Example 9.4. Using dictionaries
#!/usr/bin/python
# Filename: using_dict.py
# 'ab' is short for 'a'ddress'b'ook
ab = { 'Swaroop' : 'swaroopch@byteofpython.info',
'Larry' : 'larry@wall.org',
'Matsumoto' : 'matz@ruby-lang.org',
'Spammer' : 'spammer@hotmail.com'

}
print "Swaroop's address is %s" % ab['Swaroop']
# Adding a key/value pair
ab['Guido'] = 'guido@python.org'
# Deleting a key/value pair
del ab['Spammer']
print '\nThere are %d contacts in the address-book\n' % len(ab)
for name, address in ab.items():
print 'Contact %s at %s' % (name, address)
if 'Guido' in ab: # OR ab.has_key('Guido')
print "\nGuido's address is %s" % ab['Guido']
Output
$ python using_dict.py
Swaroop's address is swaroopch@byteofpython.info
There are 4 contacts in the address-book
Contact Swaroop at swaroopch@byteofpython.info
Contact Matsumoto at matz@ruby-lang.org
Contact Larry at larry@wall.org
Contact Guido at guido@python.org
Guido's address is guido@python.org
How It Works
We create the dictionary ab using the notation already discussed. We then access key/value pairs by
specifying the key using the indexing operator as discussed in the context of lists and tuples. Observe
that the syntax is very simple for dictionaries as well.
We can add new key/value pairs by simply using the indexing operator to access a key and assign that
value, as we have done for Guido in the above case.
We can delete key/value pairs using our old friend - the del statement. We simply specify the dictionary
and the indexing operator for the key to be removed and pass it to the del statement. There is no
need to know the value corresponding to the key for this operation.
Next, we access each key/value pair of the dictionary using the items method of the dictionary which
returns a list of tuples where each tuple contains a pair of items - the key followed by the value. We retrieve
this pair and assign it to the variables name and address correspondingly for each pair using
the for..in loop and then print these values in the for-block.
We can check if a key/value pair exists using the in operator or even the has_key method of the
dict class. You can see the documentation for the complete list of methods of the dict class using

help(dict).
Keyword Arguments and Dictionaries. On a different note, if you have used keyword arguments in
your functions, you have already used dictionaries! Just think about it - the key/value pair is specified by
you in the parameter list of the function definition and when you access variables within your function,
it is just a key access of a dictionary (which is called the symbol table in compiler design terminology).

Tuples and the print statement

One of the most common usage of tuples is with the print statement. Here is an example:
Example 9.3. Output using tuples
#!/usr/bin/python
# Filename: print_tuple.py
age = 22
name = 'Swaroop'
print '%s is %d years old' % (name, age)
print 'Why is %s playing with that python?' % name
Output
$ python print_tuple.py
Swaroop is 22 years old
Why is Swaroop playing with that python?
How It Works
The print statement can take a string using certain specifications followed by the % symbol followed
by a tuple of items matching the specification. The specifications are used to format the output in a certain way. The specification can be like %s for strings and %d for integers. The tuple must have items
corresponding to these specifications in the same order.
Observe the first usage where we use %s first and this corresponds to the variable name which is the
first item in the tuple and the second specification is %d corresponding to age which is the second item
in the tuple.
What Python does here is that it converts each item in the tuple into a string and substitutes that string
value into the place of the specification. Therefore the %s is replaced by the value of the variable name
and so on.
This usage of the print statement makes writing output extremely easy and avoids lot of string manipulation
to achieve the same. It also avoids using commas everywhere as we have done till now.
Most of the time, you can just use the %s specification and let Python take care of the rest for you. This
works even for numbers. However, you may want to give the correct specifications since this adds one
level of checking that your program is correct.
In the second print statement, we are using a single specification followed by the % symbol followed
by a single item - there are no pair of parentheses. This works only in the case where there is a single
specification in the string.

Tuple

Tuples are just like lists except that they are immutable like strings i.e. you cannot modify tuples.
Tuples are defined by specifying items separated by commas within a pair of parentheses. Tuples are
usually used in cases where a statement or a user-defined function can safely assume that the collection
of values i.e. the tuple of values used will not change.
Using Tuples
Example 9.2. Using Tuples
#!/usr/bin/python
# Filename: using_tuple.py
zoo = ('wolf', 'elephant', 'penguin')
print 'Number of animals in the zoo is', len(zoo)
new_zoo = ('monkey', 'dolphin', zoo)
print 'Number of animals in the new zoo is', len(new_zoo)
print 'All animals in new zoo are', new_zoo
print 'Animals brought from old zoo are', new_zoo[2]
print 'Last animal brought from old zoo is', new_zoo[2][2]
Output
$ python using_tuple.py
Number of animals in the zoo is 3
Number of animals in the new zoo is 3
All animals in new zoo are ('monkey', 'dolphin', ('wolf', 'elephant', 'penguin'))
Animals brought from old zoo are ('wolf', 'elephant', 'penguin')
Last animal brought from old zoo is penguin
How It Works
The variable zoo refers to a tuple of items. We see that the len function can be used to get the length
of the tuple. This also indicates that a tuple is a sequence as well.

We are now shifting these animals to a new zoo since the old zoo is being closed. Therefore, the
new_zoo tuple contains some animals which are already there along with the animals brought over
from the old zoo. Back to reality, note that a tuple within a tuple does not lose its identity.
We can access the items in the tuple by specifying the item's position within a pair of square brackets
just like we did for lists. This is called the indexing operator. We access the third item in new_zoo by
specifying new_zoo[2] and we access the third item in the third item in the new_zoo tuple by specifying
new_zoo[2][2]. This is pretty simple once you've understood the idiom.
Tuple with 0 or 1 items. An empty tuple is constructed by an empty pair of parentheses such as myempty
= (). However, a tuple with a single item is not so simple. You have to specify it using a
comma following the first (and only) item so that Python can differentiate between a tuple and a pair of
parentheses surrounding the object in an expression i.e. you have to specify singleton = (2 , )
if you mean you want a tuple containing the item 2.
Note for Perl programmers
A list within a list does not lose its identity i.e. lists are not flattened as in Perl. The same applies
to a tuple within a tuple, or a tuple within a list, or a list within a tuple, etc. As far as Python
is concerned, they are just objects stored using another object, that's all.

Using Lists

Example 9.1. Using lists
#!/usr/bin/python
# Filename: using_list.py
# This is my shopping list
shoplist = ['apple', 'mango', 'carrot', 'banana']

print 'I have', len(shoplist), 'items to purchase.'
print 'These items are:', # Notice the comma at end of the line
for item in shoplist:
print item,
print '\nI also have to buy rice.'
shoplist.append('rice')
print 'My shopping list is now', shoplist
print 'I will sort my list now'
shoplist.sort()
print 'Sorted shopping list is', shoplist
print 'The first item I will buy is', shoplist[0]
olditem = shoplist[0]
del shoplist[0]
print 'I bought the', olditem
print 'My shopping list is now', shoplist
Output
$ python using_list.py
I have 4 items to purchase.
These items are: apple mango carrot banana
I also have to buy rice.
My shopping list is now ['apple', 'mango', 'carrot', 'banana', 'rice']
I will sort my list now
Sorted shopping list is ['apple', 'banana', 'carrot', 'mango', 'rice']
The first item I will buy is apple
I bought the apple
My shopping list is now ['banana', 'carrot', 'mango', 'rice']
How It Works
The variable shoplist is a shopping list for someone who is going to the market. In shoplist, we
only store strings of the names of the items to buy but remember you can add any kind of object to a list
including numbers and even other lists.
We have also used the for..in loop to iterate through the items of the list. By now, you must have
realised that a list is also a sequence. The speciality of sequences will be discussed in a later section
Notice that we use a comma at the end of the print statement to suppress the automatic printing of a
line break after every print statement. This is a bit of an ugly way of doing it, but it is simple and gets
the job done.
Next, we add an item to the list using the append method of the list object, as already discussed before.
Then, we check that the item has been indeed added to the list by printing the contents of the list by
simply passing the list to the print statement which prints it in a neat manner for us.
Then, we sort the list by using the sort method of the list. Understand that this method affects the list

itself and does not return a modified list - this is different from the way strings work. This is what we
mean by saying that lists are mutable and that strings are immutable.
Next, when we finish buying an item in the market, we want to remove it from the list. We achieve this
by using the del statement. Here, we mention which item of the list we want to remove and the del
statement removes it fromt he list for us. We specify that we want to remove the first item from the list
and hence we use del shoplist[0] (remember that Python starts counting from 0).
If you want to know all the methods defined by the list object, see help(list) for complete details.

Data Structures Introduction

Data structures are basically just that - they are structures which can hold some data together. In other
words, they are used to store a collection of related data.
There are three built-in data structures in Python - list, tuple and dictionary. We will see how to use each
of them and how they make life easier.
List
A list is a data structure that holds an ordered collection of items i.e. you can store a sequence of
items in a list. This is easy to imagine if you can think of a shopping list where you have a list of items
to buy, except that you probbly have each item on a separate line in your shopping list whereas in Python
you put commas in between them.
The list of items should be enclosed in square brackets so that Python understands that you are specifying
a list. Once you have created a list, you can add, remove or search for items in the list. Since, we can
add and remove items, we say that a list is a mutable data type i.e. this type can be altered.
Quick introduction to Objects and Classes
Although, I've been generally delaying the discussion of objects and classes till now, a little explanation
is needed right now so that you can understand lists better. We will still explore this topic in detail in its
own chapter.
A list is an example of usage of objects and classes. When you use a variable i and assign a value to it,
say integer 5 to it, you can think of it as creating an object (instance) i of class (type) int. In fact, you
can see help(int) to understand this better.
A class can also have methods i.e. functions defined for use with respect to that class only. You can use
these pieces of functionality only when you have an object of that class. For example, Python provides
an append method for the list class which allows you to add an item to the end of the list. For example,
mylist.append('an item') will add that string to the list mylist. Note the use of dotted
notation for accessing methods of the objects.
A class can also have fields which are nothing but variables defined for use with respect to that class
only. You can use these variables/names only when you have an object of that class. Fields are also accessed
by the dotted notation, for example, mylist.field .

The dir() function

You can use the built-in dir function to list the identifiers that a module defines. The identifiers are the
functions, classes and variables defined in that module.
When you supply a module name to the dir() function, it returns the list of the names defined in that
module. When no argument is applied to it, it returns the list of names defined in the current module.
Using the dir function
Example 8.4. Using the dir function
$ python
>>> import sys
>>> dir(sys) # get list of attributes for sys module
['__displayhook__', '__doc__', '__excepthook__', '__name__', '__stderr__',
'__stdin__', '__stdout__', '_getframe', 'api_version', 'argv',
'builtin_module_names', 'byteorder', 'call_tracing', 'callstats',
'copyright', 'displayhook', 'exc_clear', 'exc_info', 'exc_type',
'excepthook', 'exec_prefix', 'executable', 'exit', 'getcheckinterval',
'getdefaultencoding', 'getdlopenflags', 'getfilesystemencoding',
'getrecursionlimit', 'getrefcount', 'hexversion', 'maxint', 'maxunicode',
'meta_path','modules', 'path', 'path_hooks', 'path_importer_cache',
'platform', 'prefix', 'ps1', 'ps2', 'setcheckinterval', 'setdlopenflags',
'setprofile', 'setrecursionlimit', 'settrace', 'stderr', 'stdin', 'stdout',
'version', 'version_info', 'warnoptions']
>>> dir() # get list of attributes for current module
['__builtins__', '__doc__', '__name__', 'sys']
>>>
>>> a = 5 # create a new variable 'a'
>>> dir()
['__builtins__', '__doc__', '__name__', 'a', 'sys']
>>>
>>> del a # delete/remove a name
>>>
>>> dir()
['__builtins__', '__doc__', '__name__', 'sys']
>>>
How It Works
First, we see the usage of dir on the imported sys module. We can see the huge list of attributes that itcontains.
Next, we use the dir function without passing parameters to it - by default, it returns the list of attributes
for the current module. Notice that the list of imported modules is also part of this list.
In order to observe the dir in action, we define a new variable a and assign it a value and then check
dir and we observe that there is an additional value in the list of the same name. We remove the variable/
attribute of the current module using the del statement and the change is reflected again in the output
of the dir function.
A note on del - this statement is used to delete a variable/name and after the statement has run, in this
case del a, you can no longer access the variable a - it is as if it never existed before at all.

from..import

Here is a version utilising the from..import syntax.
#!/usr/bin/python
# Filename: mymodule_demo2.py
from mymodule import sayhi, version
# Alternative:

# from mymodule import *
sayhi()
print 'Version', version
The output of mymodule_demo2.py is same as the output of mymodule_demo.py.

Making your own Modules

Creating your own modules is easy, you've been doing it all along! Every Python program is also a module.
You just have to make sure it has a .py extension. The following example should make it clear.
Creating your own Modules
Example 8.3. How to create your own module

#!/usr/bin/python
# Filename: mymodule.py
def sayhi():
print 'Hi, this is mymodule speaking.'
version = '0.1'
# End of mymodule.py
The above was a sample module. As you can see, there is nothing particularly special about compared to
our usual Python program. We will next see how to use this module in our other Python programs.
Remember that the module should be placed in the same directory as the program that we import it in, or
the module should be in one of the directories listed in sys.path .
#!/usr/bin/python
# Filename: mymodule_demo.py
import mymodule
mymodule.sayhi()
print 'Version', mymodule.version
Output
$ python mymodule_demo.py
Hi, this is mymodule speaking.
Version 0.1
How It Works
Notice that we use the same dotted notation to access members of the module. Python makes good reuse
of the same notation to give the distinctive 'Pythonic' feel to it so that we don't have to keep learning
new ways to do things.

A module's __name__

Every module has a name and statements in a module can find out the name of its module. This is especially handy in one particular situation - As mentioned previously, when a module is imported for the
first time, the main block in that module is run. What if we want to run the block only if the program
was used by itself and not when it was imported from another module? This can be achieved using the
__name__ attribute of the module.
Using a module's __name__
Example 8.2. Using a module's __name__
#!/usr/bin/python
# Filename: using_name.py
if __name__ == '__main__':
print 'This program is being run by itself'
else:
print 'I am being imported from another module'
Output
$ python using_name.py
This program is being run by itself
$ python
>>> import using_name
I am being imported from another module
>>>
How It Works
Every Python module has it's __name__ defined and if this is '__main__', it implies that the module
is being run standalone by the user and we can do corresponding appropriate actions.

The from..import statement

If you want to directly import the argv variable into your program (to avoid typing the sys. everytime
for it), then you can use the from sys import argv statement. If you want to import all the names
used in the sys module, then you can use the from sys import * statement. This works for any
module. In general, avoid using the from..import statement and use the import statement instead
since your program will be much more readable and will avoid any name clashes that way.

Byte-compiled .pyc files

Importing a module is a relatively costly affair, so Python does some tricks to make it faster. One way is
to create byte-compiled files with the extension .pyc which is related to the intermediate form that Python
transforms the program into (remember the intro section on how Python works ?). This .pyc file
is useful when you import the module the next time from a different program - it will be much faster
since part of the processing required in importing a module is already done. Also, these byte-compiled
files are platform-independent. So, now you know what those .pyc files really are.

Using the sys module

sys
print 'The command line arguments are:'
for i in sys.argv:
print i
print '\n\nThe PYTHONPATH is', sys.path, '\n'
Output
$ python using_sys.py we are arguments
The command line arguments are:
using_sys.py
we
are
arguments
The PYTHONPATH is ['/home/swaroop/byte/code', '/usr/lib/python23.zip',
'/usr/lib/python2.3', '/usr/lib/python2.3/plat-linux2',
'/usr/lib/python2.3/lib-tk', '/usr/lib/python2.3/lib-dynload',
'/usr/lib/python2.3/site-packages', '/usr/lib/python2.3/site-packages/gtk-2.0']
How It Works

First, we import the sys module using the import statement. Basically, this translates to us telling Python
that we want to use this module. The sys module contains functionality related to the Python interpreter
and its environment.
When Python executes the import sys statement, it looks for the sys.py module in one of the directores
listed in its sys.path variable. If the file is found, then the statements in the main block of that
module is run and then the module is made available for you to use. Note that the initialization is done
only the first time that we import a module. Also, 'sys' is short for 'system'.
The argv variable in the sys module is referred to using the dotted notation - sys.argv - one of the
advantages of this approach is that the name does not clash with any argv variable used in your program.
Also, it indicates clearly that this name is part of the sys module.
The sys.argv variable is a list of strings (lists are explained in detail in later sections). Specifically,
the sys.argv contains the list of command line arguments i.e. the arguments passed to your program
using the command line.
If you are using an IDE to write and run these programs, look for a way to specify command line arguments
to the program in the menus.
Here, when we execute python using_sys.py we are arguments, we run the module using_
sys.py with the python command and the other things that follow are arguments passed to the
program. Python stores it in the sys.argv variable for us.
Remember, the name of the script running is always the first argument in the sys.argv list. So, in this
case we will have 'using_sys.py' as sys.argv[0], 'we' as sys.argv[1], 'are' as
sys.argv[2] and 'arguments' as sys.argv[3] . Notice that Python starts counting from 0
and not 1.
The sys.path contains the list of directory names where modules are imported from. Observe that the
first string in sys.path is empty - this empty string indicates that the current directory is also part of
the sys.path which is same as the PYTHONPATH environment variable. This means that you can directly
import modules located in the current directory. Otherwise, you will have to place your module in
one of the directories listed in sys.path .

DocStrings

Python has a nifty feature called documentation strings which is usually referred to by its shorter name
docstrings. DocStrings are an important tool that you should make use of since it helps to document the program better and makes it more easy to understand. Amazingly, we can even get back the docstring
from, say a function, when the program is actually running!
Using DocStrings
Example 7.8. Using DocStrings
#!/usr/bin/python
# Filename: func_doc.py
def printMax(x, y):
'''Prints the maximum of two numbers.
The two values must be integers.'''
x = int(x) # convert to integers, if possible
y = int(y)
if x > y:
print x, 'is maximum'
else:
print y, 'is maximum'
printMax(3, 5)
print printMax.__doc__
Output
$ python func_doc.py
5 is maximum
Prints the maximum of two numbers.
The two values must be integers.
How It Works
A string on the first logical line of a function is the docstring for that function. Note that DocStrings also
apply to modules and classes which we will learn about in the respective chapters.
The convention followed for a docstring is a multi-line string where the first line starts with a capital letter
and ends with a dot. Then the second line is blank followed by any detailed explanation starting from
the third line. You are strongly advised to follow this convention for all your docstrings for all your nontrivial
functions.
We can access the docstring of the printMax function using the __doc__ (notice the double underscores)
attribute (name belonging to) of the function. Just remember that Python treats everything as an
object and this includes functions. We'll learn more about objects in the chapter on classes.
If you have used the help() in Python, then you have already seen the usage of docstrings! What it

does is just fetch the __doc__ attribute of that function and displays it in a neat manner for you. You
can try it out on the function above - just include help(printMax) in your program. Remember to
press q to exit the help.
Automated tools can retrieve the documentation from your program in this manner. Therefore, I strongly
recommend that you use docstrings for any non-trivial function that you write. The pydoc command that
comes with your Python distribution works similarly to help() using docstrings.

The return statement

The return statement is used to return from a function i.e. break out of the function. We can optionally
return a value from the function as well.

Using the literal statement
Example 7.7. Using the literal statement
#!/usr/bin/python
# Filename: func_return.py
def maximum(x, y):
if x > y:
return x
else:
return y
print maximum(2, 3)
Output
$ python func_return.py
3
How It Works
The maximum function returns the maximum of the parameters, in this case the numbers supplied to the
function. It uses a simple if..else statement to find the greater value and then returns that value.
Note that a return statement without a value is equivalent to return None. None is a special type
in Python that represents nothingness. For example, it is used to indicate that a variable has no value if it
has a value of None.
Every function implicitly contains a return None statement at the end unless you have written your
own return statement. You can see this by running print someFunction() where the function
someFunction does not use the return statement such as:
def someFunction():
pass
The pass statement is used in Python to indicate an empty block of statements.

Keyword Arguments

If you have some functions with many parameters and you want to specify only some of them, then you
can give values for such parameters by naming them - this is called keyword arguments - we use the
name (keyword) instead of the position (which we have been using all along) to specify the arguments to
the function.
There are two advantages - one, using the function is easier since we do not need to worry about the order of the arguments. Two, we can give values to only those parameters which we want, provided that
the other parameters have default argument values.
Using Keyword Arguments
Example 7.6. Using Keyword Arguments
#!/usr/bin/python
# Filename: func_key.py
def func(a, b=5, c=10):
print 'a is', a, 'and b is', b, 'and c is', c
func(3, 7)
func(25, c=24)
func(c=50, a=100)
Output
$ python func_key.py
a is 3 and b is 7 and c is 10
a is 25 and b is 5 and c is 24
a is 100 and b is 5 and c is 50
How It Works
The function named func has one parameter without default argument values, followed by two parameters
with default argument values.
In the first usage, func(3, 7), the parameter a gets the value 3, the parameter b gets the value 5 and
c gets the default value of 10.
In the second usage func(25, c=24), the variable a gets the value of 25 due to the position of the
argument. Then, the parameter c gets the value of 24 due to naming i.e. keyword arguments. The variable
b gets the default value of 5.
In the third usage func(c=50, a=100), we use keyword arguments completely to specify the values.
Notice, that we are specifying value for parameter c before that for a even though a is defined before
c in the function definition.

Default Argument Values

For some functions, you may want to make some of its parameters as optional and use default values if
the user does not want to provide values for such parameters. This is done with the help of default argument
values. You can specify default argument values for parameters by following the parameter name
in the function definition with the assignment operator (=) followed by the default value.
Note that the default argument value should be a constant. More precisely, the default argument value
should be immutable - this is explained in detail in later chapters. For now, just remember this.
Using Default Argument Values

Example 7.5. Using Default Argument Values
#!/usr/bin/python
# Filename: func_default.py
def say(message, times = 1):
print message * times
say('Hello')
say('World', 5)
Output
$ python func_default.py
Hello
WorldWorldWorldWorldWorld
How It Works
The function named say is used to print a string as many times as want. If we don't supply a value, then
by default, the string is printed just once. We achieve this by specifying a default argument value of 1 to
the parameter times.
In the first usage of say, we supply only the string and it prints the string once. In the second usage of
say, we supply both the string and an argument 5 stating that we want to say the string message 5
times.
Important
Only those parameters which are at the end of the parameter list can be given default argument
values i.e. you cannot have a parameter with a default argument value before a parameter
without a default argument value in the order of parameters declared in the function parameter
list.
This is because the values are assigned to the parameters by position. For example, def
func(a, b=5) is valid, but def func(a=5, b) is not valid.

Using the global statement

If you want to assign a value to a name defined outside the function, then you have to tell Python that
the name is not local, but it is global. We do this using the global statement. It is impossible to assign
a value to a variable defined outside a function without the global statement.
You can use the values of such variables defined outside the function (assuming there is no variable with
the same name within the function). However, this is not encouraged and should be avoided since it becomes
unclear to the reader of the program as to where that variable's definition is. Using the global statement makes it amply clear that the variable is defined in an outer block.
Example 7.4. Using the global statement
#!/usr/bin/python
# Filename: func_global.py
def func():
global x
print 'x is', x
x = 2
print 'Changed global x to', x
x = 50
func()
print 'Value of x is', x
Output
$ python func_global.py
x is 50
Changed global x to 2
Value of x is 2
How It Works
The global statement is used to decare that x is a global variable - hence, when we assign a value to x
inside the function, that change is reflected when we use the value of x in the main block.
You can specify more than one global variable using the same global statement. For example, global
x, y, z.