Undusting the blog

It's time to undust this blog a bit. Here is the list of my recent posts on LShift blog, with a word of comment:

• Introducing RabbitMQ-Status plugin. Well, this plugin is intended to help sysadmins understand what's happening inside Rabbit. Stupid, simple, robust: the way sysadmin stuff should be.
• Python Quirks. My rant about Python language. I like Python in general, but it has some dark corners and stuff that nobody uses or understands. I tried to sum up things that I would change in the language. This things should at least be better documented.
• YDB: yet another key-value database. I was disappointed with BDB/TC write performance, so decided to hack together something. I wanted to prove if I could beat them.
• Python Queue Interface for AMQP. Python Queue interface is nice. It's a shame that it's not scalable across many machines. I tried (and partially failed) to implement similar API over AMQP.
  

Paypal/Ebay are broken

I just spent more than an hour trying to buy a thing from Ebay. I'm really pissed off. In short words: Fuck you ebay.

So the story goes like that:

• I found an item
• I clicked "buy it now". So far, so good.
• They want me to register on ebay. I don't want yet-another-password and yet-another-stupid-user-name. Why the hell do I need that for? I'm trying to buy a thing, not an insurance. Why do you need my date of birth? Why do you want my phone number (landline only! mobiles not accepted!)
• We're not yet finished with the ebay account. I need to verify my account! By phone or by credit card number? Well, after 10th try I managed to give them some random fake telephone number, as I don't have landline. So I must give them my credit card number. Start counting, 1st time I entered my credit card details.
• This went smoothly. Have I already bought the item? Not really, I need to click through some stupid confirmation process. Yes I do want to buy this item. Yes I do mean that. Yes, I still haven't changed my mind.
• I almost thought it's done, how mistaken I was. Now I need to pay. Why am I able to buy thing without paying... but let's leave that question for now. 
• Click, click, yes I want to pay. I entered my credit card number (second time). Why do they need my date of birth to pay? My bank is ignoring this information, so it's absolutely useless there. Why do I have to put telephone number? Fortunately number 02012345678 works fine. Yes, do request more information from me, I'm always able to be smarter than you and enter fake data. The entropy in your database will grow. My frustration will increase.
• Ebay kindly informs me that the transaction was not accepted or something like that. Well, that shouldn't happen. Lets' try again.
• Back button, entered credit card details for the 3rd time.
• Payment is directed to Paypal, and it returns this nice 500 page: 

• 

• Checked emails. Once. Again. Nothing. Well, let's just try again.
• "We advise that you don't use your browser's back button." Fair enough, I clicked through all the ebay page to get again to payment site.
• Entered credit card details. 4th time. 
• The same paypal error. At this point I became really pissed off.
• Okay, few years ago I had paypal account, let's try to pay using that, instead of entering credit card details by hand.
• I created account in previous century, so obviously I have no idea what is the password. Clicked on "forgotten password link". 
• Click Click Click. I got email with link from them. Normally at this point recovering password story ends. But not in paypal!
• "How do you want to verify your account?". What the heck. Okay, to proceed I had to remember two of my security questions or give them the credit card number I used few years ago with that account. Of course I have no idea what my answers to the questions were.
• Next fifteen minutes I spent trying to find my old credit card, it should be somewhere...
• I was surprised to see that it should be still working, maybe I could try to pay on ebay using this card.
• Clicked through ebay again, to get to the payment site.
• Entered my credit card number. 5th time, but this time different card. Guess what happened....

• Okay, so I do need to get access to that paypal account.
• I followed the paypal-reset-password link again. Chose credit-card verification, and entered my credit card information for the 6th time.
• After that point I was able to actually pay for this ebay item using paypal. But I still was forced to enter my credit card expiry date and cvv number, so I count it as entering my credit card details for the 7th time.

Dear Ebay, I just wanted to buy a thing. I don't care about paypal or ebay accounts, and I already have forgotten passwords. I don't care if you have my addresses (every one of three I used), I feel silly when you ask me for the date of birth.  

One more thing, requiring me to give you my credit card details seven times doesn't increase my confidence in the security of your system.

Memcached protocol is not enough

For some time I'm trying to hack together a prototype of real-time-full-text-search-engine (RTFTSE!). I used a memcached-binary protocol as a communication protocol between backends and storage-nodes.

This is a pretty nice protocol and in fact there are strong reasons not to use ascii memcache protocol.

The ministry of strange syscalls

My favorite syscall today:

$ man 2 readahead
"readahead() populates the page cache with data from a file so that subsequent reads from that file will not block on disk I/O."

I don't really know when should I use that, but it sounds cool. Just an implementation of prefetching on yet another layer. Wait a moment...

"readahead() blocks until the specified data has been read. "

I'm lost. If it blocks, why not to just use read(2)?


Second syscall:
$ man 2 madvise
"The madvise() system call advises the kernel about how to handle paging input/output in the address range beginning at address start and with size length bytes"
"The kernel is free to ignore the advice."

I don't get when I should use it.


And the last one:
$ man 2 mincore
"mincore() returns a vector that indicates whether pages of the calling process’s virtual memory are resident in core (RAM), and so will not cause a disk access (page fault) if referenced."

Cool, I can check if my memory page is in swap. I doubt it can help me in anything. If I don't want my pages to be in swap I'll just use mlock(2)...


Probably there are dozens of strange syscalls out there!

GIT is ahead of SVN!

Git is the most popular DVCS right now. Google trends confirms that:



But the interesting fact is that Git is for the first time ahead of its grandpa Svn:


Though it's worrying that CVS is still alive, it should be dead ten years ago. Hopefully we can see a process of slow death, the end is inevitable:

QCon mini InThe Brain session

I'll be speaking about simplified Etherpad clone at QCon London on Wednesday, March 11th.

The presentation will take place at Skillsmatter stand (booth number 10), at 16:45.

Evserver, part3: Simplified Etherpad clone

This time I hacked together few open source projects. The result of my work, except from few minor bugs, seems to be working.


I don't know what shall I do with this project next. There are few possible options:

• throw it away and forget about it
  
• add support for markups other than reStructuredText
• implement new features, like private documents or downloading the rendered markup in different formats
  

We'll see.

For now the priority is to create a quick presentation about this project, so I could show it during the break at Qcon.

EvServer, part2: Rabbit and Comet

source

I just published the next part of the EvServer story.

I'm working at the third part. Though, my major concern is that I'm running out of proofreaders.

EvServer, Introduction: The tale of a forgotten feature

This article about EvServer doesn't really show it as an innovative piece of software. I hope that future articles will.

It seems that the most important thing in this post is a blinking exclamation mark.


BTW, ItBlog seems to be dead. For historical reasons I placed my old posts from that blog here, here and here.

I was wondering how Twitter search works

Few days ago I committed a blog post about Twitter search.

As a matter of fact, I'm not sure what is the solution to the question I asked there. On the other hand I presented some interesting (at least for me) numbers.

While writing this stuff I learned a lot and maybe this blog post will be a motivation for someone to actually create such a perfect system.

I'm still thinking about the persistence layer...