2008-01-18

django-evserver - Asynchronous server for Django

I mentioned the idea in previous post about asychronous wsgi.

Now we have working example of simple chat application. The chat is nothing special, but it's made in Django! To make this possible I created special server, django-evserver. It's quite light wrapper to libevent http layer (several hundred of lines in python/ctypes).

Using this technology it's now possible to create asynchronous WSGI applications. Currently I implemented the Comet javascript library and some wrappers for Django views, to make easier writing comet applications in Django.

Here is how example chat application evserver-chat looks like:

The code is commited to code.google.com, maybe in future I'll write some more code and documentation. For example it could be interesting how I used named fifos (mkfifo) to pass events.



25 comments:

Niels Sandholt Busch said...

Looks sweet. Thanks. I'll give it a try some day soon.

zgoda said...

Definitely worth looking. I'll try to abuse it a bit. ;)

Arnar Birgisson said...

Excellent initiative! I'll try it out for sure when I find time..

I've been meaning to write a libevent based WSGI server myself for a while. Guess now I don't have to :)

Mathieu Ravaux said...

Very interesting, can't wait to test it !

Timothy said...

Hahaha, you are aware this already exists right? When you think Asynch and python, don't you think twisted?

Twisted web2 has a wsgi server
http://twistedmatrix.com/documents/current/api/twisted.web2.wsgi.html

Glyph Lefkowitz said...

Not only does twisted.web2 support WSGI - there's already a COMET chat application demo for Twisted (using Nevow) :). It's even on Blogger.

Have a look here at Duncan's blog.

Esaj said...

The response time looks pretty impressive.

majek said...

zgoda said: Definitely worth looking. I'll try to abuse it a bit. ;)
Feel invited :)

Timothy said: Hahaha, you are aware this already exists right?
In previous post I analyzed other possibilities of comet for python. I'm not interested in Twisted for political reasons, it looks too much like Java for me. Next reason is that django-evserver is 500 python lines in server file (mostly ctypes wrapper to libevent), and about 100 lines for Django helpers. I suggest that Twisted uses a bit more LOC and that it's a bit slower. But of course the idea of async WSGI is nothing new.

Piotr Maliński said...

I played with Orbited. Their examples may have problems, but I made it working without "any" problems :) I wrote a tutorial about it :)

majek said...

Piotr Maliński said: I played with Orbited.

Nice to hear this! As far as I know, Orbited needs Pyevent, which works only for Python 2.4. But indeed it seems that we both try to achieve simmilar thing :)

Piotr Maliński said...

Python 2.5.1 and it works :) on Gentoo amd64 even :)

Michael Carter said...

Actually, Orbited 0.3.2 and 0.4.0aplah1 don't require pyevent anymore. It can be used optionally to make the network IO more efficient, but it will also work with the python-epoll module, or plain old poll or select.

majek said...

Michael Carter said: Pyevent can be used optionally to make the network IO more efficient, but it will also work with the python-epoll module, or plain old poll or select.

It means that you implemented full http stack in python which is very interesting. In django-evserver I used libevent stack, which was buggy :P

New orbited seems to be worth looking.

Bogdan said...
This comment has been removed by the author.
Bogdan said...

This looks very interesting. However, I'm having a hard time trying to host this using apache and mod_python. Is this the proper way? Can you post a (very) small tutorial?

majek said...

> This looks very interesting. However,
> I'm having a hard time trying to host
> this using apache and mod_python. Is
> this the proper way? Can you post a
> (very) small tutorial?

No. It must be executed as a standalone application. The project is currently abandoned, I'm waiting for somebody to pay me for developing this.

Jon Singler said...

>In django-evserver I used libevent stack, which was buggy :P

Buggy? What kinds of bugs did you find?

majek said...

There was a problem with chunk encoding on evhttp (http part of libevent). The \r\n was missing.

Here's my patch for this issue.

Fortunately this issue was fixed in next release of libevent.

Mokys said...

I am sorry for maybe stupid question. I can not install django evserver. I have done everything what is on http://code.google.com/p/django-evserver/wiki/INSTALLATION but it is ont working. command "python manage.py runserver" say me "Error: No module named django_evserver". How can I install it, please?

majek said...

> I can not install django-evserver.

Please try evserver instead.

You should be able to install egg from the downloads section.

Mokys said...

OK... it looks it is working... but example chat application is not working. If I understand it clear, django-evserver is "old" and chat applications won't work. Clear?

majek said...

> OK... it looks it is working... but example chat application is not working.

Yes, chat application from django-evserver will not work under evserver. Sorry. Names are similar, but interfaces aren't.

I can port the chat example to evserver if you like.

Mokys said...

> I can port the chat example to evserver if you like.

Really? It should be the best present for me :-) I have spent many many hours with "installing comet" to my server yet. Thank you very much!

Mokys said...

> I can port the chat example to evserver if you like.

When do you think you will have got it, please?

majek said...

Dead simple django chat example can be found here

Please install current evserver from svn.

You also need py-amqplib 0.6 and runing rabbitmq server.

Than you shall be able to run:

$ cd evserver/evserver/examples/django_chat/
$ python manage.py runevserver