cam macrae - blog

No mod_python

I've been using Apache HTTPD for the last decade, but since I got a VPS at Rimuhosting I've been looking for ways to reduce memory consumption. This, combined with mod_python and pyLucene having incompatible threading models, lead me to look for alternatives.

Enter Lighttpd

Lighttpd is an awesome little webserver with a very small footprint, and damn it's fast. Running a Django powered application with lighttpd requires fcgi, which in itself is a great way to save precious resource on your VPS as each HTTP server process doesn't require a Python interpreter in memory.

The simplest way to run a Django fscgi server is use a domain socket:

./manage.py runfcgi daemonize=false socket=/tmp/mysite.sock

I did this for a couple of weeks in development, but when it came to load testing the application I'd get "broken pipe" errors until every fcgi process died:

Traceback (most recent call last):
     File "./manage.py", line 11, in 
       execute_manager(settings)
   ...
   line 155, in runfastcgi
       WSGIServer(WSGIHandler(), <em></em>wsgi_opts).run()
     File "build/bdist.macosx-10.3-fat/egg/flup/server/fcgi_fork.py", line
   128, in run
   ...
   socket.error: (32, 'Broken pipe') 

I posted on django-users and asked around on IRC, but nobody shed any light on it. Got an email from a guy at Zoomr who seemed to be having the same problem.

Purely by chance, I worked out that using a TCP socket:

./manage.py runfcgi daemonize=false host=127.0.0.1 port=3033

instead of a domain socket cured the problem.

A TCP socket also has the added advantage that the fcgi server and webserver can be on different machines, which could come in handy under load.