From PHP to Python: things I wish I’d known
The story so far…
Back around 2005 I took my first leap into the world of writing useful programs armed only with my laptop, a Pentium II running Debian, and a fat book on PHP5. Though I’d taken a few classes on C++ and tinkered with BASIC on a few different platforms over the years, I’d never managed to produce anything that actually did anything practical (or, did it very well, at any rate); with PHP, though, it didn’t take long to spread my coding wings and take to the air as a novice web developer.
Before long, my skills proved useful in some critical situations at work, and my place as a web developer was cemented. My PHP code became more mature as I learned hard lessons of experience and combined them with the programming theory and computer science I was avidly consuming on the web.
Around the same time, I got interested in Python when I saw a coworker using it to build powerful desktop applications quickly and effortlessly; I wanted to round out my programming skills with a “desktop stack” as quick and simple as PHP, and Python turned out to be a good choice.
So for a few years now I’ve written mainly in these two languages, PHP for the web and Python for the desktop. As my applications have grown bigger and more complex, I began to develop a growing discontentment with PHP on a number of levels. So when people started talking about doing web development in Python, I had to see if I could make the switch.
What “they” say
Programmers who like to be vocal about things on the Internet often like to be vocal about PHP; it seems every so often someone will write an epic diatribe on what’s wrong with PHP and why everyone should avoid it. They will tell you to instead use Java, or Ruby, or Python, or whatever – just not PHP. And while I can’t bring myself to completely denounce PHP, when I’ve been waist-deep in a hacked-up PHP codebase for a while, words like “elegant”, “expressive” and “truly object oriented” stir something in me that PHP’s clumsy and inconsistent attempts at bettering itself just don’t answer.
Yet when I finally vaulted the fence in search of the greener grass, I found, predictably, that paradise has a few bald spots of its own. More importantly, though, I found that there is a significant paradigm shift that a PHP developer has to make before embracing other languages. Since nobody ever really talked about this shift, I decided to write this article to help others with the journey.
What is PHP anyway?
If your web development story, like mine, begins with PHP, you probably have a similarly skewed idea of what developing a web application is really all about. Correcting that perspective starts with understanding what PHP really is.
A templating system
At its heart, PHP is a templating system – a templating system, mind you, that’s been bitten by a radioactive web spider and mutated into a massive, object-oriented language – but still fundamentally a templating system. What I mean by this is best explained by describing what happens when a PHP document is accessed in a browser:
- The web server (Apache, e.g.) receives a request for a PHP document.
- The web server runs PHP, passing in the requested file and some details about the request
- PHP processes the requested file, dumping in any “required” or “included” files, and evaluating the “<?php ?>” bits wherever they occur.
- PHP returns the evaluated document (now pure HTML/XML/CSS/etc) to the web server, which sends it to the browser.
This is why you can stick HTML directly in a PHP file (outside the tags), and it will work just fine: a PHP file fundamentally is an HTML file (even if there’s no raw HTML in it), just one that is marked for some pre-processing1 before it’s served.
PHP is also a kind of minimal framework. It does a few nice things for you (such as parsing HTTP requests into nice arrays like $_GET and $_POST), and exposes a huge library of optional functionality like database connectivity and image processing.
It’s important to recognize this, because you need to understand that PHP by itself (apart from any third-party add-ons or abstraction layers) has solved a lot of problems for the web developer out-of-the-box. They way it solves these problems is sometimes good, and sometimes not; but in either case, it’s rarely the only way to solve them. It’s easy to take for granted that the way PHP has done something is the only, or canonical, way. Get used to the idea that it is not.
Bad, but no so bad
As I’ve said, people will bash PHP incessantly, and you can be sure they will continue to. If you can read all the way through “PHP: a fractal of bad design“2 and still feel completely good and happy about coding in PHP – well, you’ve got resolve anyway. PHP clearly has problems, ranging from niggling warts to major structural issues.
But in the end, what really matters is you, your software, and the people who use your software. PHP’s alleged shortcomings are meaningless academic drivel until they start to impact these things; and I’d guess that in a lot of cases, they don’t. Not everyone is trying to build the next big Web 3.0 killer-app-in-a-browser; some people just want to automate some things on a personal website, or build a basic CRUD system for a set of data they use at work. Don’t get caught up in the PHP hate and throw away a perfectly good tool that has solved these problems in a decently usable way out-of-the-box.
What is Python, and how do you build a web application with it?
Python is a general-purpose programming language. It has a huge set of libraries to make it useful for a wide variety of tasks, but fundamentally it’s a programming language that wasn’t designed for any specific use.
It’s also an absolutely awesome language to work with.
It’s not, however, a web framework (not even a minimal one) or a templating system; so if you intend to use it for web development on a level similar to PHP, you’ll need to add these things to Python.
I mention this because in the PHP world, there are also frameworks and templating systems – Zend Framework or CakePHP for example. Personally, I used CodeIgniter for two or three major (for me) projects, and found it very helpful… at first. After maintaining the code (and keeping CI upgraded) for a couple of years, I got disenchanted with the framework thing. I found that most of what I really wanted from a framework could be answered with a couple hundred lines of convenience functions and some careful class structure planning. So to me, “framework” initially conjured up a big bloated can of overkill.
If you feel the same way, understand that in the Python world you will need to use a framework – at least a minimal one – because (you’ll recall) you were already using a minimal framework called PHP.
The same goes for a templating system. I originally thought, “meh, why bother with a templating system?”; but really, unless you like having big non-syntax-highlighted blobs of HTML in your logic code (you shouldn’t like this!), you will want a templating system.
So, understanding this, let’s look at some frameworks and templating systems.
Python Web Frameworks
Venturing into the world of Python frameworks will immediately have you staring down the keen double-edged blade of choice. There are dozens of them, ranging from minimal and low-level to massively abstracted software suites with more gadgets than Batman’s utility belt. I have only worked with a few, so I’m going to stick with what I know here, in roughly the order that I tried them.
mod_python and python server pages
Before I really “got” python web development, I of course went to it with the mindset of a PHP developer; this sent me straight into the arms of mod_python and python server pages (PSP). This setup allows you to do with Python pretty much what you do with PHP: you write a python script, and Apache executes it in-place whenever it gets a GET or POST request for the script. The script in question can either be pure python, or an html file with PSP embedded (PSP is basically a templating system; to use it, you just drop in tags (delimited by <% %>) much like you do with PHP).
At first this seems pretty cool; but after a while, you realize that you haven’t really improved the structure or security of your application over PHP, and you’ve traded away all the really nice web-oriented conveniences of PHP just to get a language with slightly (OK vastly) better syntax. It seems kind of a wash, and in the end I never saw much incentive to switch based on mod_python.
Apparently, the Python & Apache communities concur, because as of 2010 the mod_python project was abandoned. No great loss, as far as I can tell.
If you say “Python Web Framework?” in a room full of people, chances are they’ll just look at you funny; but if anyone actually does respond, they’ll probably say “Django”. Some consider it to be the one-and-only Python web framework worth talking about, and the banner under which all such development should rally.
Django is what you’d call a full framework; it’s got pretty much everything built-in and ready to roll: templating, database abstraction/ORM, sessions & security, and even its own web server.
Despite the features and popularity, Django didn’t really get traction with me for a few reasons:
First, because it’s not really my style to use tools that are “everything you could ever possibly need right out of the box”; I tend to prefer more minimal, modular tools that I can build on easily.
Second, because it’s one of those frameworks that has you start out a project by running a special “init” command which flomps out a tree full of generated code, into the midst of which you’re to write your application. I could be wrong, but in my previous experiences, this sort of thing means painful upgrades down the road as you surgically inject code updates into your app.
Finally, (and I could be wrong here), it seemed to me that Django wanted to be in charge of the database layout and code, and wouldn’t work well with an existing database. Since I am very often tasked with writing alternate front-ends or reporting tools for existing (and often read-only) databases, this doesn’t work for me.
I decided to give up on Django after a few days of tinkering, and went back to PHP.
After Django, I was attracted to CherryPy on a couple of counts. First, it doesn’t generate a big tree of boilerplate code – it’s just a library that you import. More importantly, it isn’t attached to a particular templating system or database layer, so you can use whatever you want (or nothing at all). I was still in the “I don’t need no stinkin’ template system” mindset when I tried it, and the project I wanted to do didn’t talk to a database at all; so this minimal framework was ideal for me.
I managed to produce a nice utility which we still use at my work, but my experience was somewhat mixed. Chalk it up to my inexperience with Python web development, but I had basically two problems with my CherryPy project:
- There were at least two or three ways to do certain things, like setting configuration options, and the documentation would mix and match them. The same method didn’t always work for me, either; this made my code feel inconsistent.
- Many things in the program require some automagical combination of function and variable names, such as session authentication. I had a lot of trouble getting these to work properly, and the debugging messages were not helpful.
Maybe with more time and experience, and longer study of the documentation, I could get along with CherryPy; but at the time, I had work to do and PHP was ready to roll up its sleeves.
Lately I’ve discovered Flask. Like CherryPy, it’s also a simple library that just has to be imported, and allows you to use whichever templating system or database layer you like (though it seems well-integrated with jinja2). So far I get along pretty well with Flask; it feels a little more straightforward than CherryPy, without a lot of the automagical things based on function or variable names. I’ve had some problems with the way it parses complex form data (doesn’t like nested arrays, apparently3). It’s also helped me realize my need for a templating language, and I learned pretty much all I needed to learn about Jinja2 in roughly ten minutes.
Granted, I don’t have any Flask-based code in production at this point4, so I may still be on the hype curve. I’m working on my second Flask-based app right now, and still pretty pleased.
Python Templating systems
I don’t have a ton to say about templating systems, because I’ve only used Jinja2 so far; but I will say this for the benefit of PHP developers like me who think they don’t need a templating system: do yourself a favor and use a templating system. Remember that PHP is a templating system, so it isn’t as if you haven’t already been using one (you probably just didn’t realize it). Don’t do what I did in my CherryPy project and just put your HTML in strings and use printf()-style substitution. Yeah, it works, but it’s ugly, you don’t get proper syntax highlighting, and it mixes presentation with logic.
Don’t stress about which one to use, either. They’re all pretty much the same, just different syntax. I picked Jinja2 because it’s well integrated with Flask and purported to be fast. But really, there’s no wrong answer here; it takes about 30 minutes tops to figure out a new templating system.
One nice aspect I found with Jinja2 (and others probably have similar features) was the ability for templates to inherit from other templates. So I can create a basic template for all pages with some named blocks, then have specific other templates inherit that template and only add to the specific blocks they need to change. This is a nice change from the way I usually did it in PHP, where I’d have functions or class methods to produce various parts of the page (header, navigation, content, etc).
Some general notions you need to get familiar with
So you’ve got your framework and templating system picked out, but there are now some concepts you’ll want to get used to and some bad habits you’ll want to break in order to produce a Python web application.
The web page is not STDOUT
In PHP, if you want to put something on the web page, you just echo it. Since we’re just pre-processing a document in-place, anything PHP spits out ends up on the resulting page. This isn’t the best style of PHP coding, but now and then it’s how you solve a coding problem, and admittedly a lot of us PHP coders debug this way (using echo or var_dump() at any arbitrary points in the code to see what’s going on).
Python frameworks like Flask and CherryPy don’t operate this way; instead, you write functions that return a chunk of HTML, which the framework packages into an HTTP response. So unlike PHP, if you just “print” something, it won’t end up on the resulting web page. Instead, it ends up wherever STDOUT has been directed5
This wasn’t so much of an adjustment for me, because this is generally how I wrote PHP applications anyway: as a set of functions that returned chunks of HTML. For some PHP coders, who may be accustomed to just echoing data as they get it, this is a very foreign approach.
Earlier I described how a web server goes about handling a request to a PHP file; basically, if I browse to http://example.com/lib/docs/help, the web server would look for <webroot>/lib/docs/help/index.(something). If that “something” happens to be “php”6, then PHP gets the file, does its thing, and we’re all happy. If there isn’t an appropriate index file there, you get a 404 or maybe 403 error. Bottom line, if you want to make the URL available with PHP, then (short of doing some magic in the web server configuration, e.g. mod_rewrite), you need to create a PHP file at that URL.
With modern Python frameworks, it’s not like this. Making a particular URL on the server return a page of HTML doesn’t require that actual files exist at the path; rather, the framework has a system to map URLs to functions or classes in your program. This is generally called “routing”.
So, in the example above, making that URL work on a Python-framework site would not require you to create any files under /lib/docs/help, or even create those directories under the webroot. It just requires that your code maps the path ‘/lib/docs/help’ to a function or class that returns a valid response (a chunk of HTML, or an HTTP response object, e.g.).
Some PHP frameworks or applications let you do something like routing (a.k.a. “pretty/simplified/clean URLS”), usually by using mod_rewrite to hide a request for some actual php file.
Before I got into doing web applications with Python, I confess that I didn’t really understand HTTP all that well. If you only work with PHP, you probably know that information can be sent to the server as a “GET” or “POST”, and you know the difference in terms of how it looks in the URL bar. It turns out there’s a good bit more to HTTP than just GET and POST, and if you’re working with a Python framework there’s a lot of benefit in understanding it all.
It’s not that Python frameworks don’t abstract HTTP; it’s that they abstract it differently than PHP does – and frankly, I think they abstract it in a cleaner and more natural way. For instance, while PHP makes the contents of an HTTP request available to your script through a series of global arrays (e.g. $_GET, $_SERVER, etc) or function calls (e.g. getallheaders()), Python frameworks tend to give you some kind of request object that encapsulates the entire HTTP request.
If you consider the fact that your Python framework and application are handling many of the things that PHP just let the web server handle, it makes sense that you’re going to be living a little closer to HTTP; don’t let this scare you though. The more I’ve learned the more I start to think that HTTP itself is actually simpler than the mess that PHP abstracts it into.
If you’ve used one of the more well-known PHP frameworks (like CodeIgniter, e.g.) you’ve probably come across the term “MVC”7. MVC is a software design strategy that cleanly divides your application into three parts:
- The Model, which is your data model and handles all read/write operations to the database or data store
- The View, which is the interface the end user actually sees and interacts with
- The Controller, which manages interactions between the View and the Model
If you haven’t used a framework, or disciplined yourself to using the MVC pattern already8, you’re in for a shock. Python frameworks, so far as I have seen, all pretty much enforce the idea of MVC on one level or another. This is ultimately a good thing for your code, and without patterns like this it gets difficult to scale to a real application; but it’s a real learning curve if your usual MO is to sandwich database queries and business logic between large chunks of HTML ad-hoc.
Planning and design
OK, the “real programmer” crowd out there may sneer a bit at this point, but if you’re going to move off PHP you need to prepare to do a little more planning when you create a web app. If PHP excels at anything, it’s in allowing you to just sort of hop in and code by the seat of your pants and build something huge without a lot of advanced planning, sticking code wherever it seems to fit to get the job done.
The python frameworks I’ve encountered enforce a bit more rigidity than this. Unlike PHP, you aren’t just tromping through a script from top to bottom, so you can’t just define things willy-nilly and expect them to be in scope when you need them. Similarly, your templating language is merely a templating language, not a full-blown programming language; so you can’t just stick a bunch of function calls and variable assignments in your template because it feels good.
In addition, frameworks like Flask use a lot of design patterns and abstract constructs whose application isn’t immediately obvious to people who don’t get deep into software design theory. Properly applying these constructs takes some contemplation about what your application is doing on a more abstract level.
This is ultimately good for you, because it forces you to design a better and cleaner application than you might have done in another language. It absolutely requires a different mindset and approach, though.
Downsides to Python
In addition to the need to navigate the paradigm shifts I mentioned, there are some downsides and pitfalls to using Python for web development, and I think it’s fair to mention them.
Deploying your PHP application is as simple as dumping your .php files into a folder under the webroot9. Deploying a Python web application is a little more involved, and there are multiple ways of doing it each with its own pros and cons to weigh.
And of course, your web host may not support Python, Flask, or any of the other Python libraries you need to run your application; I can almost guarantee that the company who hosts this website doesn’t. If you were thinking of writing a CMS in CherryPy for people to use on their $1.99/mo shared web host, you’re wasting your time.
If you’re into high-level abstractions that take all the icky SQL out of your code and replace it with objects and method calls, Python has you covered well with ORM layers like SQLAlchemy. If you’re like me, and you actually prefer to write your own SQL, the situation is a little different.
PHP has a nice database abstraction layer called PDO, and if you aren’t using it, you need to quit typing “mysql_” and start using PDO. PDO is a single library with consistent classes, methods, and properties no matter what database driver you’re using.
Python, on the other hand, has the DB-API, which is a specification for writing database libraries for Python. Instead of one library with drivers for different databases, Python gives you product-specific libraries that (mostly) adhere to the DB-API. In theory, this should be as nice as PDO, but in practice it’s more chaotic. Some libraries extend DB-API with extra functions, and within the spec there is flexibility in how certain things are implemented (e.g., parameter style for parameterized queries). I found this problematic when (for example) I ported a Flask application written for psycopg2 (a popular PostgreSQL library) to cx_Oracle (the one and only Oracle DB-API library) and had to adjust a lot of code10 due to differences in interpretation of the spec.
Of course, someone out there is screaming at me now to use ORM, but I’m just not there yet.
Frameworks like Flask or CherryPy seem to assume that you are actually writing an application, not just an arbitrary collection of unrelated web content. This is great if you actually are writing an application, but not everyone is. Sometimes you’re just creating a page that needs to do a little computational work before it displays, or a form that needs some simple processing on submission.
If you have a website like, say alandmoore.com, which contains several subsections of marginally-related content, much of which is legacy and doesn’t need to be touched, using PHP is pretty frictionless. If I want to add a new page which has nothing to do with the rest of the site, but which needs to do some special processing before it displays (like, say, scanning a directory of MP3 files and displaying ID3 information), this is as simple as writing a new .php file and dropping it into a directory.
If my website were handled by a Python application, this would get trickier. I’d either have to write and deploy a new WSGI application (which seems a lot of work and overhead for a single page), or I’d have to add a new function to my existing application (complicating and potentially destabilizing my whole site).
The current curse of the Python universe is this whole Python 2 to Python 3 transition that’s been going on for the last few years. In the web development space, there are many popular libraries and frameworks that don’t yet support Python 3; Notably Flask, Python Imaging Library, and Django11, though hopefully this will change soon.
What this means for new Python coders is that you need to keep coding in Python 2, but you need to do so in a way that will be easily portable to Python 3 in the future. Which basically means you need to learn both versions. Eek. If you’re not already invested in Python, this can be a deterrent.
Do you need to switch?
I’m not going to be one of those bloggers who insists that you need to switch away from PHP to Python or anything else. PHP isn’t going away any time soon, and for a lot of applications it’s perfectly adequate and easy to ideate. I will say, however, that taking the time and effort to embrace a language like Python and the disciplines of the more popular frameworks is beneficial to any web programmer.
I’d suggest sticking with PHP in the following scenarios:
- Your pages are mostly HTML with just a little bit of dynamic content here and there
- Your work needs to be deployed on cheap shared web-hosting
- You don’t think of what you’re writing as an “application”, just “web pages”
If you’re doing serious web development work, and you have control over the production server, then I think it’s well worth the effort to use Python for your next big project. I will most likely continue to do a lot of work with PHP for legacy projects at work and for clients; but as much as possible, I’ll be moving to Python from here on out.
1 Note the official PHP backronym is “PHP Hypertext Preprocessor.
2 I mention this rant a few times because it’s made waves in the web development community over the last year or so, and makes a lot of good points. For the record, I don’t agree with everything in it (some points seem a little contrived or unfair), but a lot of the criticism is spot on (like the inconsistencies in naming and parameter ordering) and convinced me to try harder at branching out. I think any PHP developer should read that rant with an open mind.
3 To be fair, I fixed this issue with about 10 lines of python.
4 This isn’t Flask’s fault, I have a reporting application ported from PHP mostly done, but held up by one of our vertical app vendors.
5 Which, when you’re using the built-in development server, is usually to the terminal from which you ran the application. Which is pretty wizard, really.
6 or any other extension which the web server is configured to interpret as PHP
7 Model View Controller
8 Read: “If your code is haphazardly littered with calls to mysql_query()…”
9 Assuming PHP is already set up and configured, which is pretty trivial in these days of cheap web hosts and easy Linux distributions.
10 I mean Python code; it’d be natural and expected to have to adjust SQL code between products, of course.
11 Python 3 support is currently experimental