The pythonic way to do international development

Here I describe how the stages of learning to program in python mirror the levels of design complexity that international development agencies deploy to tackle problems like sending children to school or ending global poverty.

First – some context about what python is and why it matters.

In the future we will all be program managers

I’ve been teaching myself how to program in python for the last 2 years now. Why? Because I needed to accomplish tasks that demanded more than just excel and whatever else is out there for the “everyman” to use. Now I cannot imagine working without it. I believe that in a generation, “literacy” will be defined by one’s daily use of a programming language like python.

Do you remember the era of calculators, when logarithms and exponential functions were the sole purview of engineers, statisticians, and other “data experts”? Today we treat statisticians as specialists, but that is changing because the world is inundated with data, and linear inferential statistics is just one way to approach the problem. The other ways all require some programming prowess.

And before that, “human computer” was as a career. From the seventeenth century until the era of digital computers, groups of mostly women were paid to manipulate large sets of numbers in a methodical fashion. Some mathematicians were known to even “marry their computers.”

So extrapolating from history, everyone will be a programmer by the time I have gray hair.

pythonSo what is python anyway?

Python is more than a programming language; it is the language of interoperability.

python-import-bacon

Basic scripting doesn’t need to be elegant, it just needs to work. And python provides a “low floor, high ceiling” space for people to program solutions to everything. “Low floor” means that anyone can get started with just a few lines of code, and most of these lines read like plain English. “High ceiling” means that as people use python they discover ever more elegant ways to do the same thing, often with huge performance gains. Even the process of writing error-free code can be done in much less time, because python tolerates errors and ambiguity better than most languages.

Strive for pythonic over perfection

For something to be “Pythonic” means it embodies the simplicity, readability, and elegance of the python way of doing things. What this design principle sacrifices in absolute performance (computing speed) it more than makes up for in time saved writing, debugging, and improving the code later. A long time ago a smart fella named Guido realized that most problems aren’t solved in one step, but require iterative cycles of testing, learning, and redesign. Therefore each cycle is faster if the code is pythonic. Some programmers even speak of the “Zen of Python:

zen-of-python-poster-a3

The only Zen aphorism I quibble with is “in the face of ambiguity, avoid the temptation to guess.” I believe the next generation of pythonic programming will apply a lot of heuristic guessing to unlock the tools to the masses.

Making code understandable to everyone is a subversive attack on the monopoly of the programming world. For if something is intuitive, we can teach ourselves. We can invade the lucrative kingdom of the programmer. Our children will grow up programming their phones the way we program our music stations to feed us what we want today. We will still need paid professionals, of course, but they will be called upon to manage the most complex tasks.

If python was a religion, it would be Unitarian Universalism, as explained here:

Python is simple and unrestrictive; all you need to follow it is common sense. Many of the followers claim to feel relieved from all the burden imposed by other languages, and that they have rediscovered the joy of programming. But there are some who dismiss it as a form of pseudo-code.

In contrast, the Religion of C has been described this way:

C would be Judaism – it’s old and restrictive, but most of the world is familiar with its laws and respects them. The catch is, you can’t convert into it – you’re either into it from the start, or you will think that it’s insanity. Also, when things go wrong, many people are willing to blame the problems of the world on it.

And java (and javascript):

Java would be Fundamentalist Christianity – it’s theoretically based on C, but it voids so many of the old laws that it doesn’t feel like the original at all. Instead, it adds its own set of rigid rules, which its followers believe to be far superior to the original. Not only are they certain that it’s the best language in the world, but they’re willing to burn those who disagree at the stake.

Python feels empowering to the novice, yet welcoming to the journeyman, because it has a sense of humor about itself. It is, after all, named after Monty Python.

xkcd-python

The philsophy behind C is very different. Its name comes from the idea that computer “machine code” is an A-level ASSEMBLY language. Assembly language assigns values to registers on the CPU microchip itself.

Machine code looks like this. Clearly, not meant for human consumption EVER:

8B542408 83FA0077 06B80000 0000C383
FA027706 B8010000 00C353BB 01000000
B9010000 008D0419 83FA0376 078BD98B
C84AEBF1 5BC3

This function in 32-bit x86 machine code calculates a Fibonacci number.

Assembly code looks like this:

SET r1, 10
SET r2, 1
LOOP1TOP:

SUB r1, r1, r2
CMP r1, r0
JMP NEQ, LOOP1TOP
SET r2, 20
LOOP2TOP:
LOAD r1, X
CMP r1, r2
JMP EQ, LOOP2END
...
LOOP2END:

In the 1960s, 1970s, and 1980s people developed various B-level languages that would be more readable but directly translate to operations performed on bits in microchips. Most of these were still unintelligible to the masses. C is the third generation, or C-level, attempt to bridge machine code with English in an efficient way.

C++ code looks like this:

for (i=10; i > 0; i--) {
... // loop1 body
}
while (x != 20) {
... // loop2 body
}

Python code looks like this:

data = list()
for x in range(20):
data.append(x)

But if you are familiar with python, it is just as “correct” to write the same code in short hand:

data = [x for x in range(20)]

That’s how I would write the program. In one line of code, called a list comprehension. It is easy to understand, concise, and translates to machine code almost as well as C, because Python’s interpreter is actually written in C.

And while the top five programming languages in the job market are all derived from C (java, C, Objective-C, C++,  and C#), there is another story here. Many people using python didn’t learn it to get a job, but rather to eliminate work from their job. The language of interoperability ought to transcend and absorb the best of all the other specialized programming languages, which python does. Its modules incorporate features of C, Java, Perl, Haskell, Lisp, and other langauges. Cython and Jython are essentially versions of python designed to make writing C and Java easier.

jythoncython

While web browsers don’t read python (they do read php and javascript), you can write websites in python.

bitnami-djangostack cherrypy

And over half a million $25 circuit boards based on python have been sold:

raspberry_pi_inside

And scientists? Yeah, we use python in our data analysis.scipy_conf_logo

Test first, second, third… scale later

Python is a prototyper’s dream. Its ease and readability come from being a ‘dynamically typed’ language, unlike C. Python doesn’t require variable declarations and a variety of other structural rules found in other languages to run. You can get started in python with a tiny bit of knowledge. As your projects get bigger and more complex, you can later impose structure on your code to avoid errors. And when you’re serious about performance, you can simply use the more advanced Python modules which “wrap around” high-performance C functions.

I believe the logical evolution of this strategy is to a visual graphical programming interface where people can connect chunks of code together like cogs and widgets in a machine, but underneath they are organizing functions in real code they never see. In theory, a 5 year old could write a smart phone app this way. In fact, kids are already using RasberryPi boards to do this.

So what does pythonic international development look like?

The point of this lengthy introduction is to introduce a set of design and program evolution principles that make sense in international development.

  1. Amorphic scripting: At the beginning – you’ll write your first program as a linear script with a “just give me the answer!” mentality. You won’t care about structure, legacy, or efficiency,  just what works.Surprisingly, quite a number of non-profits start out with an amorphic scripting mentality to their work. A person sees a crisis and take charge to alleviate suffering. And they can see that they are making progress, up to a point.At some point, usually 2 years into running a new organization (or 6 months for a new programmer) this approach starts to break down. Work is taking more work to make fewer gains. Staff feel like they are tackling the same problem repeatedly.This leads to the next level in program design: Modules
  2. Functional programming: After a few tries at programming with a sequence of instructions, most people realize the power of organizing their bits of code into functions. Modules are collections of functions with a related purpose. Python uses a ‘namespace’ to keep all these sets of functions organized. So if you were to “import antigravity” as the comic shows, it would keep these functions separate from some other “fly” function you might be using in the same program. By the way – DO try to import antigravity in python; it works.Nonprofits come to a similar realization and tend to hire full time staff members with specific tasks, such as grant writing, marketing, program managing, advocacy, and youth engagement. They tend to hire the monitoring and evaluation officers last. These appear to offer less bang for the buck until you realize that you are trying to do solve a complex problem. Without monitoring and evaluation tools and methods, you begin to lose track of what worked and what didn’t. Your versions of social programs get muddied in your memory.The same happens in programming. The next stage is using a code repository like GitHub for version control and collaboration.
  3. Collaboration and knowledge management: Repositories (or repos for short) allow programmers share code, or fork the same project into multiple subprojects while tracking changes to the original idea. GitHub has ~4.6 million repositories and 2.4 million users; that’s more than 1 repo for every community group in the world.The nonprofit world has tried to create “communities of practice” and “best practices” and write training manuals, do capacity building, and establish guidelines – but none of these come close to the simplicity of version tracking and knowledge dissemination that GitHub has provided the open-source computer programming community.And in fact, the nonprofit world’s whole approach to this problem appears to be on the wrong track, according to a recent PlosOne paper: Searching the Clinical Fitness Landscape. I’ll be re-blogging this paper in plain English soon, but what they show is that randomized controlled trials (RCTs) and centralized knowledge dissemination do not improve patient outcomes as well as fostering many decentralized smaller quasi-experiments to improve practices.  (or… Many people trying to achieve slightly better practices beats everybody trying to adopt a few best practices).Forking a GitHub repo and publishing a tweaked copy of an idea yields better and better code in the long run. And in fact, python was created that way. Most of python’s power is in the modularity of all the functions. You can run it quick and efficient and sparse or you can import some Goliath modules like scipy or nltk (natural language tool kit) that give you the full power to analyze data, albeit with higher memory usage.

    But even the ability to share code pales in comparison to the power of real-time error checking, which is the next conceptual leap that programmers make.

  4. Iterative design and testing – eventually a programmer will have created something useful enough that it is time to publish a product. Some tool, software, website, or analysis will require a bit more maintenance. Strangers are using your tools and messing up your code, finding all sorts of bugs as they do things you never anticipated. Your new feature development grinds to a half as you spend all your time maintaining the code wasteland you once so lovingly believed was crisp and elegant.This is where I am. Luckily my friend turned me on to py.test and TRAVIS.CI.These tools allow you to write a series of debugging scripts that will automatically try to break your code each time you push updates to your GitHub repository. TRAVIS.CI emails me whenever it finds a bug in my code. Py.test allows one to write the “business requirements” of a program before even writing the program itself. It is a different way to manage the problem, but ultimately a better one.Nothing in international development seems to mirror this feature. If it did, it might be some combination of real-time feedback and peer group benchmarking on progress towards stated goals.
  5. Agile, iterative thinking – and along the way many programmers realize that solving problems is going to be an iterative process, so they adopt agile.In international development, I’d summarize this as going from an idea to a working prototype in 14 days. Any idea too big to test in 2 weeks is too big to succeed, period. Either your resources are too scarce to handle the task, or the task is too big and complex to be feasibly managed. Most big ideas can be broken down into 14-day prototype tests. Agile says this is the best approach.

    Test your idea in a small chunk, then refine it and reexamine your assumptions. Let that TRAVIS.CI real-time bug checker crunch on your prototype while you do these tests, and you’ll find yourself becoming very good at “programming” whether you mean coding or running social programs. Both require some flexibility to adapt.

  6. Upgrading the language to unlock new capacities: A slim few programmers realize that they have been constrained by the language they were using to write code, so they begin working on a better way to write code. Python was one of those examples, but it will not be the last one.Python was a step forward because it allows people to be competent problem solvers regardless of their level of expertise. It emphasizes readability and elegance so that people minimize the time they spend debugging or describing the problem. International development is sorely lacking in people who have the audacity to rewrite the coding language it speaks. It doesn’t speak the same language from one group to the next, and much of it looks like machine code to me:

    8B542408 83FA0077 06B80000 0000C383
    FA027706 B8010000 00C353BB 01000000
    B9010000 008D0419 83FA0376 078BD98B
    C84AEBF1 5BC3

References

http://blog.startifact.com/posts/older/what-is-pythonic.html

http://python-history.blogspot.com/2009/01/brief-timeline-of-python.html

http://blog.aegisub.org/2008/12/if-programming-languages-were-religions.html

http://wiki.answers.com/Q/Why_name_c_language_is_named_as_c

http://en.wikipedia.org/wiki/Type_system

http://www.cprogramming.com/langs.html — comparison of programming language styles

http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html — index of job growth by programming language

http://img4.joyreactor.com/pics/post/9gag-auto-264324.jpeg — comparing programming languages if they were used to write an essay

I choose programming plagiarism over incomprehensibility.

programming languages

Advertisements

5 thoughts on “The pythonic way to do international development

  1. Wow! This is a really good piece. I appreciated the Python part more than the international development analogy, mostly because I am a programmer by trade. I do often think about real world problems in terms of programming methodology, though.

    I only recently tried out Python. Having heard so much about it, I rewrote a Linux account provisioning tool. I had written the tool in PERL, but PERL’s limited native data structures and clunky object-oriented support led me to Python. The project was a lot easier, and somehow more fun, to write in Python, but later attempts to write parsing-intensive tools in Python led me back to PERL.

    The breakdown of C, Java, and python was very insightful. However, as this article appears to be intended for the less technically inclined, it may be misleading to cast C and Java in such a harsh light. Just as with PERL in my example, most languages have their place. On that note, I would be interested to hear your thoughts on other high-level languages like Go or Scala if you ever get to trying them.

  2. I agree Rob. I did not intend to disparage C and Java as the world’s software runs on them. I wanted to emphasize that python is designed for people like me – a non-programmer working in an unrelated area (Science, Intl Devt, etc) with less of a need for raw power and efficiency.

    I’ve just recently heard of Go (by google) and it looks to overlap with python’s intended audience.

    Scala – if you check my link about which languages have growing job markets, this one’s persistent popularity appears to baffle the programming job prognosticators. Otherwise, I know nothing about it.

    In Nairobi, php and ruby-on-rails dominate. Not because they are the “best’ but because there is a critical mass of local people to help each other be good at it. I cannot over emphasize how much the community-of-practice around any task makes everyone better at it, no matter what language you choose.

    But for me, python is the most fun, and has the fun people you WANT to learn from.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s