Dan Kegel

It shouldn't be a problem. Checking the classic C10K article at http://www.kegel.com/c10k.html ... > In 1999 one of the busiest ftp sites, cdrom.com, actually handled 10000 clients simultaneously through a Gigabit Ethernet pipe HTTP isn't that heavier (if at all) than FTP, so 100 visitors for static content shouldn't be a challenge for 2008 hardware

I hope they bring back Battlezone - Activision rebooted it in 1998 (https://en.wikipedia.org/wiki/Battlezone_(1998_video_game)) A great single player and co-op "real time strategy" game that ran on a simple protocol called ANET (http://www.kegel.com/anet/) (http://www.strickleton.com/anet/) Most servers are dark now but there's still a great community keeping this game alive. It had very robust tools for building and sharing your own maps for deathmatch and strategy campaigns. One of the first games I ever fell in love with.

My experience with Janus is that it will not scale at least for the sole reason of using poll not epoll for the I/O (http://www.kegel.com/c10k.html#nb.poll). When I tried to use it for the last time (1-2 years ago) it also had severe stability issues but maybe that improved over time. Maybe drop me an email at [email protected] so we can discuss the matter in greater detail?

Do you know if the most optimal FreeBSD zero copy support is only available for Tigeon NICs? I couldn't quite figure it out from reading the following pages: - http://www.freebsd.org/cgi/man.cgi?query=zero_copy - http://www.kegel.com/c10k.html#zerocopy - http://people.freebsd.org/~ken/zero_copy/

Anyone else notice Yahoo's new Global News Anchor Katie Couric is one of those helping spread doubts about vaccines, the HPV vaccine in this case[1]. HPV is responsible for more than 25,000 cancers in the US each year[2] and around 6,000 deaths[3]. Worldwide figures are probably 15 times greater. [1] http://www.latimes.com/entertainment/tv/showtracker/la-et-st-katie-cou… [2] http://www.cdc.gov/cancer/hpv/statistics/cases.htm [3] http://www.kegel.com/hpv/deaths/

If you want to learn about some more advanced strategies, you may also benefit from this classic: http://www.kegel.com/c10k.html

I'd never heard of the C10K (http://www.kegel.com/c10k.html) but I love scaling issues and this looks to be some fascinating reading. Thanks! EDIT Interesting annecdote... Cal Henderson is the author of "Building Scalable Websites" http://www.amazon.com/gp/product/0596102356 and has worked in "the trenches" as the lead engineer of FLICKR and developer at B3TA. Apparently his newest company http://tinyspeck.com/ is using node.js for their game engine. Assuming he's planning on scaling to a reasonable size, that seems to be a pretty resounding endorsement that there's at least something going for it. I mean ... that guy's got a bit of experience in working at scale.

OK, you hooked me with the title. But "FreeBSD + Erlang" was kind of a dissatisfying reason for how you achieved it. Would love to hear more details! How far we've come since http://www.kegel.com/c10k.html

Even with Python (CGI backed by Apache), you can scale with threads. The issue of scaling with threads vs. events is a pretty hot debate, and I think the author sets up this kind of criticism by addressing it poorly. Personally, I fall into the event-driven camp, because it involves the operating system as little as possible (only file descriptors). The C10K link has a great overview of the threaded approach (http://www.kegel.com/c10k.html#threaded). Read through the rest of the page for a discussion on the pros/cons of other approaches.

Also do not forget classics - http://www.kegel.com/c10k.html ^_^

The broadcasting loop shouldn't take too long... I'm pretty sure that the socket.send() calls are non-blocking. I could be wrong, I haven't tested my (toy) chat server that thoroughly. :-P In general, the single threaded evented model tends to scale better than the process/thread per connection model for large numbers of connections. Having to context switch between 10,000 threads/processes typically means the OS spends a lot of time doing context switching instead of IO. I'm not sure how Clojure implements agents, it may already handle having 10,000 concurrent agents efficiently. I'd be curious to know. Also, there's a great article about network programming, which you may have read. It's called "The C10K Problem", which talks about scaling to 10,000 concurrent connections. http://www.kegel.com/c10k.html .. really good article, well worth a read if you haven't already.

It was definitely like that. One landmark that comes to mind is the Microsoft "Halloween documents" attempting to slow the rise of open source. https://en.wikipedia.org/wiki/Halloween_documents Microsoft also advertised NT and IIS as being faster than Linux (implicitly admitting that Linux was worth considering) which motivated a lot of back-and-forth "benchmaxing". http://www.mindcraft.com/whitepapers/first-nts4rhlinux.html https://www.kegel.com/nt-linux-benchmarks.html Sun was also a huge loser to open source and x86 but instead of fighting it their CEO Jonathan Schwartz embraced it. In retrospect this "Glasnost" approach didn't help Sun survive. I remember the Sun fans used to complain that Linux/x86 was winning in benchmarks but Sun was worth paying N times more (of your employer's money) because it just felt faster and more robust. Parallels to today are left as an exercise for the reader.

edit: see replies to my comment. I mistook the Hyper pull request being merged for being in the latest release, but it isn't. Rocket uses Hyper for its HTTP server. So I checked to see if the Hyper HTTP server was really production-ready. Particularly, if it could handle async IO / solve the C10K problem[0]. It looks like Hyper implemented async IO[1], so it should be adequate for production use in this regard. This is great news for users of Rocket because it means you don't need a separate HTTP framework to run your web service, as you would with, say, Django + gunicorn. Sure, it's a dependency, but in this case, having a production-ready HTTP server out of the box is really nice! Having said that, is there any literature on Rocket/hyper for security? The production-ready HTTP server is great, but it also means it has to be prepared to deal with certain security issues, like listening on 0.0.0.0 and handling file uploads. You could put HAproxy/nginx/whatever in front of it, but I think Rust has the potential to supply / manage all of this within one unit and simplify the stack / attack surface area. [0]: http://www.kegel.com/c10k.html [1]: https://github.com/hyperium/hyper/pull/778

There are really two parts to point 2. 2.a. is about performance. And while not specifically about C++ this article covers the main idea http://www.kegel.com/c10k.html 2.b. is about avoiding deadlocks and race conditions. Threaded programming in C++ is dangerous. For years I would have said, that I could avoid deadlocks and race conditions by virtue of being a good programmer. However, I've debugged enough of my own code to know this is not always true. If you only use threads for data driven computation, then you greatly reduce the risks. On the other hand if you have multiple threads running around accessing anything and everything in your program then you need a lot of thread locks which is tedious to implement and not well supported in C++ (vs. say the synchronized keyword in Java), does not perform well and still it is very easy to make mistakes. The point is that there is no reason to let threads run willy nilly because you can do event driven programming in C++ (libev, libevent, libuv) and most programs (or rather most parts of a program) are not computationally intensive. edit: Here's a great resource on the topic of threading. From the SQlite FAQ "Threads are evil. Avoid them." http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf

I understand your point... in terms of numbers Java does attract careerist programmers simply by critical mass. Although I would argue that language is actually now Javascript but thats another argument entirely. The point I was making was that there is a massive and grossly unfair generalisation solely about Java developers being made here and all over HN... that they don't really love programming, or want to write the best code possible. You might feel the language precludes that but then I would beg to differ and we are back to blub allegories. By your own figures (which I don't agree with) the top 20% of a big number is a large amount of talented programmers you have excluded from your search because you cant get past the crud. Developers who have taste, know the JVM inside out and can write tasteful, readable, lightweight (yes) and highly performant code. Code that runs everyday in the big houses such as Google, Twitter, Linkedin... I've interviewed many Java, Python and Javascript developers and our screening process helped to weed out the very hopeless, not perfect but most. First a scan of their resume - is it generic, do they have experience in complex projects, a github account, a blog, contribute to open source, opinionated? Second a real programming test that can test style, consideration of performance, typical gotchas. This should be automated so you can outright reject incorrect output... not fizzbuzz. Other people more experienced on me have commented…

Black Hat USA 2009: Realistic probabilities in modern signals intelligence[1] attempts to provide a more realistic prediction of what could be possible. The paper is structured in a similar way to how Physics and Technology for Future Presidents[2] arrives at the physical limits of remote sensing satellites. comScore qSearch reports that US Internet search traffic to the top 5 search engines (Google, Yahoo, Bing, Ask, AOL) is on the order of 20 billion per month or 7,600 searches per second averaged across the day[3]. You'd only need a few cheap off-the-shelf servers to process and store all of that information for eternity. See [4] for example performance figures for inserting data to a PostgreSQL database on a single server (not even using fancy sharding techniques). The C10K problem[5] is also worth noting in this context. The reason I use search traffic as an example is that it is very high value -- it doesn't just capture what people are reading or doing, it captures what they're seeking to read and do (motivations, desires). It's also a very compact data stream and requires little or no processing and analysis. DNS query traffic, registers of source and destination phone numbers, etc also provide cheap, valuable data. Whether or not the NSA is looking at this kind of data is of little concern when compared to what the commercial world could use this data for. The NSA has a strong incentive not to use their power too widely or they will tend to lose any advantage…

Take a look at the c10k problem, a classic resource for this problem domain: http://www.kegel.com/c10k.html You probably don't want to spawn one thread per connection, but get notified by the operating system when one of a bunch of TCP socket connections has an event for you to handle. You can achieve this by the kernel calls epoll, poll, select, kqueue. For Python, I recommend taking a look at tornado (https://github.com/facebook/tornado), an asynchronous socket handling framework. In particular the "io-loop", which you could use for handling the connections. As for serialization protocols, don't roll your own but consider JSON for maximum flexibility, and protocol buffers for maximum efficiency. For communication between servers consider well known libraries and protocols such as AMQP (eg RabbitMQ) and Redis pubsub. Be sure to read up about Linux kernel settings to improve the number of maximum open sockets, etc. Here are several good readings referenced: http://serverfault.com/questions/10852/what-limits-the-maximum-number-… Also, you'll want to get familiar with some tools you'll be using a lot, such as lsof and netcat. About the infrastructure design: you'll most likely end up with several different server applications and possibly a number of simultaneous running instances. Consider Amazon EC2 micro instances or something similar to start with. I'd recommend to experiment with building several prototypes before the real…

1) Learn some languages that force you to think differently, even if they aren't of an immediate use in the industry. I'd suggest starting with Scheme (Little Schemer + SICP) and then learning an ML family language (Haskell or OCaml). A language that incorporates elements of logic programming (Erlang or Prolog) would be useful too. You may not be lucky enough to use these languages in your day-to-day work, but there are lessons you can take home with you e.g., minimizing mutable state. 2) Remember to work on projects that are something more than algorithmic puzzles. Algorithmic puzzles are fun, great and are brain gymnastics, but you should also learn how to write low-level socket code (Richard Steven's "UNIX Network Programming" books are excellent in that respect), deal with concurrency (switching languages again, Java Concurrency in Practice is a great introduction). 3) Participate in open source projects. Dan Kegel has an excellent guide on how to contribute: http://kegel.com/academy/opensource.html Even if your contribution to an open source project is small, it will require reading and understanding a great deal of non-trivial code. Finally, as moonpolysoft said, don't call yourself a ninja or a rockstar.

I googled 'epoll "O(1)"'. You're the first hit (congratulations), followed by some random person on stackoverflow, the documentation for a CPAN module, the repetition of that documentation on github and in various package repos, and some random bloggers and mailing list responders who are either just referring to the work that you have to do to find an active fd when the wait call returns, or are confused. The man page says nothing about O(1): http://www.kernel.org/doc/man-pages/online/pages/man4/epoll.4.html The original epoll writeup says nothing about O(1), and shows graphs explicitly for dead connection scenarios (iow, it never claims better performance per active fd): http://www.xmailserver.org/linux-patches/nio-improve.html The Dan Kegel writeup that's pretty much universally cited on this issue says nothing about O(1): http://www.kegel.com/c10k.html#nb.epoll The link you threw up yesterday that "flat out gets it wrong" says nothing about O(1) and simply shows epoll doing exactly what it was designed to do (again, it never claims better performance per active fd), including graphs that actually do show poll outperforming epoll with small numbers of dead connections: http://lse.sourceforge.net/epoll/index.html Can you show any authoritative sources who claim epoll is O(1)? If not, then yes, you built a fluffy little strawman and took him down hard. I have no problem with your work on superpoll. I'm skeptical it will help much, guessing there will be…

I have been an OpenBSD and FreeBSD user, as well as minor contributor, since the mid-90s. All the answers here are good, and I would like to add: * FreeBSD is on the cutting edge of performance. Because it is a total OS (as opposed to just a kernel), performance improvements are made and integrated into the binary packages and ports build options of common packages. FreeBSD was one of the first operating systems to implement kqueue, which is now available on OS X. This was part of the reason why for years FreeBSD dominated high-performance web serving. If you asked this question 5-10 years ago, the answer would simply be 'FreeBSD performs a lot better than Linux with web serving', but that is not as true today since Linux has caught up in the 2.6.x branch (with epoll). For more info see the classic: http://www.kegel.com/c10k.html The good servers like nginx and varnish take advantage of these performance benefits that FreeBSD offers. (there was a similar system here with the virtual memory manager, while the Linux kernel struggled with this issue ~01-02 FreeBSD was miles ahead). FreeBSD is the starting and development point for a lot of new technologies. 2. Everything is done the BSD way, from directory layout to the rc.x startup system. It is the same and uniform and integrated with the kernel and doesn't vary across releases. There is a single reference handbook, and POSIX compliance. I have always preferred the BSD way of file system layout, and that is what I have…

> Running a thread per client is famously known as The C10K Problem. Gonna have to be that guy: the C10K problem was "how to serve 10000 clients from a single server" specifically in the 2000ish time-frame. The article links to the C10K problem page, which discusses the tradeoffs involved in at least five high-level strategies[1], only one of which is the strategy of running a thread per client [2]. [1] http://kegel.com/c10k.html#strategies [2] http://kegel.com/c10k.html#threaded

As a backend web developer you can learn about the difference in pre-fork and polling model in web servers, which interconnected with c10k http://www.kegel.com/c10k.html This will give you ability to reason about the web server configuration you want to use. But both fork(2) and epoll(7) [kqueue(2), iocp] would stay at low level relatively of place where you operate. Don't know what to say about fronted though, but there are probably some new points of view on JS you can get by implementing it as an interpreter in courses like crafting interpreters.

Your comment makes nginx sound like some fly-by-night server that only achieved its performance by making lots of tiny-yet-dangerous "optimisations" like this one. More likely it is an omission, which could be rectified with a warning or failure running nginx -t (verify configuration). The actual performance comes from an architectural choice between event vs process based servers, as detailed in the C10k problem article. [1] [1] http://www.kegel.com/c10k.html

Concurrency isn’t a “nice layer over pthreads” - the most important thing is isolation - anything that mucks up isolation is a mistake. — Joe Armstrong Threads are evil. https://www.sqlite.org/faq.html#q6 https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf Nginx uses an asynchronous event-driven approach, rather than threads, to handle requests. https://aosabook.org/en/v2/nginx.html http://www.kegel.com/c10k.html

I'm referring to a similar overlapping problem called C10k. http://www.kegel.com/c10k.html This was fixed at the OS level. But the usage of new system calls involved fundamental shifts in basically every language to account for the new paradigm.

How about the marvel that was Walnut Creek's cdrom.com that served 10,000 simultaneous FTP connections back in 1999? [1] I was always blown away by how much more efficient FreeBSD's network stack was compared to Linux at the time. It convinced me to go FreeBSD-only for a few years. [1] http://www.kegel.com/c10k.html

Wikipedia says Nginx started in 2004. If you look at the state of things for threads and other "lots of sockets to deal with" things in that timeframe, you can see multiprocess was probably a safer choice, especially if you intended on running well on a variety of Unix or Unix like OSes. This page has some of that state captured pretty well: http://www.kegel.com/c10k.html

I guess this is the modern equivalent of the c10k problem http://www.kegel.com/c10k.html

> Yes, you can with [...a helpful example...] Some days it's nice to have a lucky 10k problem (https://xkcd.com/1053) in place of a c10k problem (http://www.kegel.com/c10k.html). Thanks.

Maybe everyone needs their ansi.sys moment though. Back in the early 1990's, I, and probably half the programmers using DOS, rewrote the dos ansi.sys driver/TSR because its was _SLOW_. mostly IIRC because it was calling the BIOS text output, but also because it was a pretty solid implementation of the ANSI terminal/esc sequence "standard". So, the first half took about an hour or two to handle output (just write a character at the right location in the framebuffer) and get the INT/TSR boilerplate right. Then for the next 20 hours fight with all the edge cases the cropped up with corrupted text in various TUI programs that were calling the DOS string functions. Then as the size+perf approached NANSI.sys (http://www.kegel.com/nansi/) give up... Learning experience.