Jeremy Dunck

This is the system I use. I've tried lots of different methods but this is what works for me. The best thing about it is that it's not based on heavy process (Note: YMMV because this was written before the recent Gmail redesign). https://xph.us/2013/01/22/inbox-zero-for-life.html

I use this system and it has transformed my business and personal life: http://xph.us/2013/01/22/inbox-zero-for-life.html

There's some discussion of this in the notes from their release announcement: http://xph.us/2011/04/13/introducing-doozer.html

Heroku use it for their Doozer datastore which manages their distributed grid thingy. Info here: http://www.theregister.co.uk/2011/05/05/google_go/print.html http://blog.golang.org/2011/04/go-at-heroku.html http://xph.us/2011/04/13/introducing-doozer.html And of course google use it internally.

Actually, the blog entry mentions it, just not the article: http://xph.us/2011/04/13/introducing-doozer.html

Here is the actual Doozer introduction by its creator: http://xph.us/2011/04/13/introducing-doozer.html

Very interesting post. I'd advise you to go read Keith Rarick's introduction to doozer http://xph.us/2011/04/13/introducing-doozer.html. I think this could gain traction as a credible HA data store. As stated paxos is part of the core of doozer, its something that's also used by google's lock service chubby which is used in systems like GFS, bigtable and map reduce.

People who like the simplicity of the Memcached protocol will enjoy hacking a client library for the super fast Beanstalkd message queue. http://xph.us/software/beanstalkd/ Sure, it's non-persistent and similar things could be hacked with the viciously fast Redis DB or other Key/Value DBs, but it's fun to telnet into a message queue and issue plain ascii commands :-P

A recent article on a job queue, Beanstalkd: http://www.topbit.co.uk/serendipity/archives/21-Doing-the-work-elsewhe… Beanstalkd is described at the link below, and was originally designed to help handle the Facebook Causes app, with a claimed 9.5 million users. http://xph.us/software/beanstalkd/

beanstalkd (http://xph.us/software/beanstalkd/) also has similarities to this, and for non-Django / simpler needs, it may be better. It's basically memcached repurposed into a queue server. A "task" would be equivalent to a script which only looks for jobs in a certain bucket (or "tube" as they're called). You can run as many clients on as many machines as you like. Obviously, since it is memory-based, you'll lose the queue in the event of a system crash. That being said, as a rabid Django user, this is definitely going into my bookmarks!

A pretty similar design was my first try for a simple whiteboard system but switched to another misuse of memcahced with prepend and append. Timing out of old messages is a nice feature of this solution. These are dirty hacks comparing to a "proper" solution but sometimes they are OK enough. Beanstalkd has a memcached like protocol but designed for simple queues. It's a fine system but I'm missing the timeout feature. http://xph.us/software/beanstalkd

A few systems, always a work in progress. - https://xph.us/2013/01/22/inbox-zero-for-life.html for managing email - bitlbee + weechat for managing social messaging. Slack for work. - A custom deployment of https://github.com/gollum/gollum for managing all notes, personal and work related. Can easily share Markdown files with teammates / friends. https://github.com/zachlatta/mullog is code for custom deployment. I usually edit locally using vim and have tons of shell scripts specific to note management to automate file creation, renaming, and git usage. This is my most used life system and I'm almost constantly using it. - Arch Linux with i3 and tons of custom scripts for managing how I use my computer. Have heavily customized my machine in https://github.com/zachlatta/dotfiles. It's difficult to predict the future and unlikely that someone else will build a tool that will always meet my personal needs, especially as they change over time. Better to use systems that lend themselves to customization so I can always build systems that work for me – hence heavy reliance on scriptable programs.

Please see: http://xph.us/2011/04/13/introducing-doozer.html

He suggests the following message queues: http://www.rabbitmq.com/ - first version 8th February 2007, but the first version not to have "alpha" or "beta" status was 1.5.1 released 21st January 2009 http://memcachedb.org/memcacheq/ - version 0.1.1 released 26th November 2008 http://www.ejabberd.im/ - not really a message queue http://xph.us/software/beanstalkd/ - first public release 11th December 2007, hit 1.0 28th May 2008 http://activemq.apache.org/ - not sure when it was first released but the mailing list goes back to December 2005 The first public release of Starling (Twitter's first custom message queue) was 10th January 2008. Presumably they had it running internally for a while before they released it. From this, we can see that when they built their own pretty much the only realistic open source option was ActiveMQ, which can hardly be described as a light-weight solution (not to mention it still doesn't have a stellar reputation under high loads). When the alternatives aren't rock solid yet, rolling your own (where at least you understand all of the code and how it works) seems like a perfectly practical alternative.