Willy Tarreau

By having two Directors and using Linux HA (High Availability) tools to monitor and take over. http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/LVS-HOWTO.failover.html#d… Also check out HAProxy http://haproxy.1wt.eu/

I recommend starting from reading the HAProxy site. http://haproxy.1wt.eu/ It has a ton of information about building a load balanced infrastructure using HAProxy. This article helped me too: http://horms.net/projects/has/html/node6.html Read some architecture articles from this site: http://highscalability.com I am also planning on doing that, so once suppliers allow multiple machines I'll write a blog post about I had to do.

Re: HAProxy from http://haproxy.1wt.eu : Update [2012/09/11] : native SSL support was implemented in 1.5-dev12. The points above about CPU usage are still valid though. Wow, I've been out of the loop for a while then. Last time I used HAProxy I was using stud (https://github.com/bumptech/stud) to terminate.

The Varnish authors have specifically stated they didn't want to introduce SSL into their process because of the added attack surface that adds. [0] If you want SSL for Varnish, use something like Pound [1], stunnel [2], or HAProxy [3] to do the SSL termination and pass it off to Varnish... I'm partial to Pound because it is lightweight and doesn't try to do any caching, it simply terminates the SSL and passes the request on. [0] https://www.varnish-cache.org/docs/trunk/phk/ssl.html [1] http://www.apsis.ch/pound [2] https://www.stunnel.org/index.html [3] http://haproxy.1wt.eu

HAProxy[0] has fit the bill for me in many instances. [0] http://haproxy.1wt.eu/

http://haproxy.1wt.eu/ shows homepage from 2009 where as http://haproxy.1wt.eu/?foo gives latest homepage

This is nice to see. I wonder how it compares to HAProxy http://haproxy.1wt.eu/ which also works with SSL and Secure WebSockets (wss://).

Managing load with thin workers isn't very hard...haproxy [http://haproxy.1wt.eu/] makes it pretty easy to setup rather complex load distributions (certainly more complex than random!).

Yeah, you want leastconn + app/router affinity. Affinity is the statement that all of your requests for an app go through one router (to avoid distributed connection state tracking). In the past, I've accomplished this by having the next layer up consistent hash the api keys onto the router list. If you don't control the top layer (ELB), you need to add a dumb layer just for the hashing. HAProxy works great for this extra layer. In practice, all you end up doing is adding a "balance hdr(host)" directive (see http://haproxy.1wt.eu/download/1.5/doc/configuration.txt) to get the hashing right, and you're spending <1ms inside HAProxy.

Quick note: 1. I'm not affiliated with either these guys or HAProxy. 2. The HAProxy guy(s) posted this too - see the site (http://haproxy.1wt.eu/).

On a simmilar note, HAProxy can proxy http traffic or just tcp traffic, which you can use to cluster your service and provide a little redundancy. http://haproxy.1wt.eu/

> If you start with GPL stuff (and in my experience most of the really good open source software is GPL) Much as I prefer the GPL to BSD licences, that sounds odd to me, unless you mean for pretty small values of most. Some best-in-class Free software more or less off the top of my head, that isn't GPL: apache (most everything, but lets stick with httpd and solr) nginx openssh freebsd openbsd openssl ghc go (golang) plan9 PostgreSQL redis riak elasticsearch clang/llvm v8/chrome If you add inn LGPL the list grows, perhaps notably to include erlang and to a certain extent haproxy (I didn't realize licensing had changed, the rationale may be interesting in this context[1]). I guess there are a couple of ways to productize GPL software -- one is the RedHat/Canonical approach: use (rather large) resources on testing and creating a coherent whole, including branding (arguably RedHat does more work here than Canonical -- but Canonical certainly aren't the first to try to value-add on top of Debian). The other way is to sell an appliance of some kind. You could say the kindle and even Android fall under this category (as does haproxy, I guess). > In such a scenario, any business that needed an off-the-shelf 3D editor or image editor but also developed in-house software (for example, Pixar) would find it in their interest to patch bugs and generally improve the software that they use, even though they don't sell…

Whoops! I definitely said SPDY and definitely meant the PROXY protocol (http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt) You can setup stunnel to terminate SSL then append this line to the request that's sent to HAProxy, which will then add an X-Forwarded-For header from that info. This may be relevant to your interests, though: http://www.igvita.com/2012/10/31/simple-spdy-and-npn-negotiation-with-…

Nginx/HAProxy/Pound probably don't care about disk access if they are balancing. http://haproxy.1wt.eu/news.html shows that he got nearly 10Gbps on "cheap" hardware (excluding the donated NICs)

I'd suggest writing an add-on or utility for a popular data-managing server. You'll learn a lot and perhaps create something many others can use without having to get in too deep. Here are some suggestions. I personally like the databases listed but I put the cache stuff first since thats what you seem to like. Cache / Proxy tools: http://www.squid-cache.org/ http://haproxy.1wt.eu/ Web Servers: http://www.apache.org/ http://www.lighttpd.net/ http://nginx.net/ http://www.acme.com/software/thttpd/ Databases: http://www.postgresql.org/ http://www.sqlite.org/ Version Control Systems: http://subversion.tigris.org/ http://www.selenic.com/mercurial/wiki/ http://darcs.net/ http://git.or.cz/