Thursday, April 17, 2008

Writing Python modules in C

Python just like other scripting languages is interpreted meaning that a program written in python is processed by an interpreter that encodes it to computer understandable machine code during runtime. The interpreter overhead does reduce its performance over machine compiled languages like C & C++. One advantage though is that programs written in interpreted languages are portable as long as the platform has the corresponding interpreter installed.

Thinking forward:
--- "Is there a way to write an application that should perform as fast as possible but at the same time take the advantage of python rapid development?"

--- "I have a project that uses a protocol that can only be written in C. Is there a surefire way to make my life easier?"

...and the answer to both of the questions is "yes" there's a way.

Most of the modules that comes with python is actually written in C or C++. They are compiled as a shared object library and it does makes applications using it run faster. It's just like writing small and fast C modules and gluing them together with a high-level language. One popular example of this technique is the game Civilization IV which uses python as its interface layer on its C++ game codes and today I'll teach you how to do that. :)

First you need to have these necessary stuff installed in your system:
gcc - GNU C compiler
python-dev - python include files (you can download this on their website)

For this tutorial, I wrote a very simple and useless python extension that takes two integer values and returns the sum:

#include <Python.h>
int a, b, sum;
static PyObject *
Add(PyObject *self, PyObject *args)
if (!PyArg_ParseTuple(args, "ii", &a, &b))
return NULL;
sum = a + b;

return Py_BuildValue("i", sum);

PyMethodDef adder_methods[] = {
{"add", Add, METH_VARARGS, "Add two integers"},


(void) Py_InitModule("adder", adder_methods);

Here's a quick run through:
Line 1: Include Python header
Lines 3-4: Create a function named Add that accepts and returns an abstract Python Object
Lines 6-7: PyArg_ParseTuple says that the function will require two Integers denoted by "ii" if the requirement is not met return NULL (which causes an error)
Line 8: Our shameful adder logic
Line 9: Py_BuildValue converts the C integer variable sum to Python integer denoted by "i"
Lines 12-15: PyMethodDef defines the functions available in this modules. As you can see it defines a function name "add" that points to our C function "Add".
Lines 17-20: Initializes our modules and assigns it with a name "adder"

To be able to use this in python, we need to build it first. The easiest way to do this is to use python distutils. Create a file named

from distutils.core import setup, Extension
import sys

setup(name = "adder",

version = "0.1",
ext_modules = [Extension("adder", ["adder.c"])])

Then build adder by this chain of commands:
# python build (compiles it with gcc)
# sudo python install (installs it to python site-packages directory)

And there you have it. Your own python module written in C. Cool isn't it?

Sadly this is a blog, if I just have an actual web page I could explain this much further. For now, if you want to know more about the Python.h stuff that I used in this program, head over to python/C API reference manual

This tutorial actually leads to my next blog which is about how do SMS (Short Messaging Service) actually work and at the same time give you a nice python extension that I wrote (SCTP sockets in python!). I hope that I helped somebody out there, Til next blog! ^_^


spot said...

That was really heplful! Now I can run code I have written for my simulations in C and use matplotlib to plot all the data!

tschwaerzl said...

thank you very much! helped me a lot!