Three Reasons Why You Shouldn't Write Your Own Web Framework

Categories: python, web, programming, framework, turbogears, django, zope

Jeremy Jones has some thoughts about Django and Rails. He also asks the question: "...what if re-creating existing code sometimes isn't as evil as we've been taught?" Good question to ask; I'd respond in the comments if I had an O'Reilly account, but I don't, so here goes:

Motivating Story

Let's say you're a developer and you're looking to build a whiz-bang dynamic content website. Your weapon of choice is Python, so you start looking at the various Python web frameworks. After losing many hours and half your hair examining the 31 flavors (and growing!), you decide to look into one or two in detail. Say you look at Django and TurboGears. Django has about 60% of what you need, and TurboGears has about 75% of what you need. At which point you say "Python web frameworks all suck!", you write your own, and you port your site to it. (Not to name names, or anything... ;-) ) So you end up a) examining the frameworks, b) writing your own new framework, c) actually constructing your site in your new framework, and d) promoting your new framework to others.

Why is this a bad thing?

I've got three reasons that come to the top of my mind. But first, let me give you an alternative story. Rather than writing your own framework, you decided to add the 25% that TurboGears lacked. You either became a TurboGears developer with source tree access and updated the source directly or sent your patches to someone who would integrate them later. You then ported your site to the (possibly customized) TurboGears framework. So why was the first scenario (where you "rolled your own") bad?

1. You wasted your time

I'm not saying that you could have avoided all the work of developing your own framework by extending someone else's. I'm just saying you could have avoided some of it. There's some overhead in reading the other framework's code and grokking the design, as well as in actually modifying the framework by implementing the functionality you need. But I contend that it's a lot less work than you would spend inventing your own framework.

Then there's the problem of bugs. In the 75% of TurboGears that you re-implemented, you wrote bugs. You probably revised your initial design a few times. This is all work you could have avoided if you'd chosen to contribute to an existing project rather than rolling your own.

Then there's the work you did promoting your framework to others. If you'd enhanced TurboGears (or Django, this is just an example), you could ride the free wave of publicity that comes with a well-known project.

2. You wasted other people's time

This really comes from promoting your framework. If you want to keep everything to yourself, you won't waste anyone's time but your own. But when you make it available to others, you generate more work for the next developer trying to decide on a framework. (And because of programmer style, he's just about as likely to decide that your framework sucks as any other.) But he had to do the research. If you were on his "short list", he had to spend a decent amount of time weighing the benefits of using your framework. And if he decided not to go with you, that's time that is (mostly) wasted.

3. Your work has diminished impact

You did some clever things implementing your framework. You might have used a new abstraction that no one has considered before. Maybe you created a templating framework with features that 70% of developers want, and no other framework provides. You want to contribute back to the community. Well, by creating a new framework, the size of your community is a lot smaller than it could be. Contribute code to an existing framework, make it even better, and all its users benefit from your work. "Roll your own," and you'll end up pulling a few developers from using other frameworks to using yours, but you'll probably still have a smaller impact than if you'd contributed to an existing framework.

The dynamics of markets usually end up promoting only 1-3 really dominant players: "gorillas" and "chimps", to borrow from Geoffrey Moore. All the "monkeys" then end up competing for the scraps left by the others. This is true in "real" marketplaces, and it's true in the marketplace of ideas. Most people have some resistance to change, and a great many don't want to spend any more time than necessary making decisions. Where is all this going? Realistically, you're not too likely to become the gorilla if you roll your own framework. And even if you manage to, you've diminished the impact of the previous gorilla's contributions to the state of the art. In the FOSS world, gorillas are can actually be a net positive for everyone, and (I believe) should be encouraged. If OpenBSD, FreeBSD, and NetBSD were stronger competitors to Linux in Free kernels, FOSS kernels would run on a lot fewer machines, there would be less hardware support, and things would in general be a lot worse for everyone involved.

Why do people continue to re-invent the wheel?

So why do we have such a proliferation of web frameworks? Here are some guesses:

• It's really easy to build a new framework - Titus Brown discusses this a bit. The "pain threshold" in rolling your own is so low that you might as well do it.


• Writing code is more fun than reading code - This is one that bites me all the time. You convince yourself that you could write what you need faster than you could grok existing code and extend it. Maybe so. But if you're like me, you'd end up grossly underestimating the amount of time needed to test and debug your new design, not to mention the design changes you make as you develop it. (Maybe call that "grokking your own design" ;-) )


• It's cool to have your own framework - OK, there are personal benefits to rolling your own. You can blog about it. Other people can blog about it. People get to know your name. Self-promotion may sound crass, but it's nice to be recognized. And it helps getting a job. For the benefit of the state of the art, though, I'd still rather you contributed to an existing codebase.


• You want something with a fundamentally different architecture than anything that's out there - I think this is perhaps the first really valid reason to roll your own. Zope should not be the only framework to choose from. It works for a lot of people. But it has a baroque architecture compared to pretty much any other Python web framework. I'll go out on a limb and say that most people who think everything else is fundamentally wrong for their needs are just trying to justify a decision that was actually made because of one of the above three reasons. It's better to find a mainstream framework you can live with than invent a "perfect" framework that no one else can.


• The existing project manager won't accept your help - I guess sometimes this happens. It seems to have happened with XFree86. OK, go ahead and roll your own. If the manager's truly impossible to deal with, you might even fork the project. But I think this, like the reason above, it a lot less likely than many think.
  

Am I a hypocrite?

In this post, yeah, a little bit. As more of a user of frameworks than an author, I haven't contributed code to anything (yet). I use a "chimp" language (Python) rather than a "gorilla" (Java, or Perl or PHP for dynamic languages). I've rolled a lot of my own code because I was too lazy to grok someone else's. Hopefully I'll get to the ideal in this post eventually. But I am trying to head in (what I think is) the right direction.