2007-11-02

Libevent under python

So I want to write asynchronous tcp/ip server in python.

I really hate overblown twisted. The thing I like most in Python is simplicity and easiness to read. Well, twisted in my opinion doesn't have this attributes. It's of course my personal feeling, mostly because I don't know twisted. But such statements are making me sick, at first sight I really don't understand what are they for (from core twisted documentation):


internet.TCPServer(7779, IFingerFactory(f)).setServiceParent(
service.IServiceCollection(application))

I remember such cascades from Java rather than Python and I dislike Java especially for this way of programming.

Getting back to point. If not twisted than what? Lets try libevent. Stable C library, which is a production standard now. Memcached is using it, Tor is using it, IO also. It must be really professional asynchronous library.

There are also some other async linux libraries, like liboop. Wait a moment, libevent 500k google hits, liboop 25k. Sorry, the only asynchronous library for linux is libevent for now.
(btw. Liboop's api is in my opinion totally broken)

Are there any python wrappers for libevent? Google says about two: libevent-python and pyevent. Both seem abandoned.
I don't like exceptions inside libraries, so it didn't took long to eliminate pyevent:
    Traceback (most recent call last):
File "./tights.py", line 37, in
main()
File "./tights.py", line 34, in main
event.dispatch()
File "event.pyx", line 262, in event.dispatch
TypeError: exceptions must be strings, classes, or instances, not type

Update: Michael Carter writes in comments that this bug is the fault of pyrex rather than pyevent and that it shouldn't happen in python2.4(I use 2.5). I'll have to take a closer look at pyevent again.

Libevent-python seems to be working. Unfortunately this wrapper removed the concept of 'userdata' passed to callback in favor of class-based approach. The difference is that callback is not called with 'userdata' but it's called on specific class instance, so callback has 'self' object. I personally prefer the 'userdata' approach, but it's quite easy to switch.

Here's an example of simple libevent-python server, which is just printing every data it receives (something like standard echo_server.py example from libevent-python, but simpler)
#!/usr/bin/python
# -*- coding: utf-8 -*-
import socket, signal, libevent

def callback_interrupt(signum, events, event_obj):
libevent.loopExit(0)

def callback_onconnect(fd, events, event):
# hack to avoid passing sock_srv from main()
sock_srv = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)
sock, (host, port) = sock_srv.accept()
conn = Connection(sock, (host, port))

class Connection:
def __init__(self, sock, addr):
self.sock = sock
self.addr = addr
self.sock.setblocking(False)
libevent.createEvent(sock, libevent.EV_READ, self.callback_onread).addToLoop()

def callback_onread(self, fd, events, event_obj):
buf = self.sock.recv(4096)
if not buf: # just disconnect
self.sock.close()
return
# Yeah, print!
print "%r %r" % (self.addr, buf)
# reuse current event
event_obj.addToLoop()

if __name__ == '__main__':
# bind.
sock_srv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock_srv.setblocking(False)
sock_srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock_srv.bind(('0.0.0.0', 1111))
sock_srv.listen(5)

libevent.createSignalHandler(signal.SIGINT, callback_interrupt).addToLoop()
libevent.createEvent(sock_srv, libevent.EV_READ|libevent.EV_PERSIST, callback_onconnect).addToLoop()
libevent.dispatch()


UPDATE #1:
Here seems to be another way of connecting libevent with python, using raw ctypes: http://william-os4y.livejournal.com/3718.html.


2 comments:

Unknown said...

I think the error you got with pyevent is due to a bug in pyrex not the pyevent source. The error doesn't happen in python 2.4, only python 2.5 + pyevent 0.3.

Gary van der Merwe said...

Two other python libraries for libevent, but that also provide async io:
http://eventlet.net/
http://www.gevent.org/