Simon Marlow

This is in a similar space to reactive programming, but isn't reactive at its core. The best explanation of the monad we use developed is described in the paper here: http://community.haskell.org/~simonmar/papers/haxl-icfp14.pdf Query sharding is at the data source layer, which Haxl doesn't delve into. It's up to each data source integration with Haxl to do the appropriate routing/etc. Hope you find it useful!

Thank you for clarifying! I did not know that all functions in Haskell are considered curried. My surprise stemmed in part from reading a bit about "arity" from [1]. It's interesting how the theoretical model of Haskell--"all functions in Haskell take just single arguments"--differs from implementation, where, for functions of known arity, GHC in particular does not actually "follow the currying story literally" [2]. [1] http://hackage.haskell.org/trac/ghc/wiki/Commentary/Rts/HaskellExecuti… [2] http://community.haskell.org/~simonmar/papers/eval-apply.pdf

I would rather recommend chapter on deterministic concurrency in this paper: http://community.haskell.org/~simonmar/par-tutorial.pdf [In particular `par` and promises.] This solution is quickest to debug, and most efficient. One can also use multiprocessing-like approach: http://docs.python.org/2/library/multiprocessing.html My last 15 years of practice indicate that other approaches lead to unnecessary complication, and are less likely to be effective (threads and semaphores in particular.)

> I don't think the second claim is true at all, though. Lightweight threads have been extremely successful in Haskell, Erlang, etc, and not just on microbenchmarks. They're mostly successful in languages that GC/heap-allocate all stack frames (paying the costs that come with it). Rust uses the machine stack so the benefits come with some very significant drawbacks, most notably unpredictability of performance that comes from stack thrashing. Large mallocs are slower than small mallocs in general, but on the typical sizes you need for machine stack segments you probably aren't going to hit the malloc implementation's free list anyway, so you might as well just allocate up front. If you GC stack frames, though, you just bump allocate in the nursery, at the cost of greatly increased GC pressure (but it makes task spawning much cheaper). There's some interesting discussion on this near the end of Simon Marlow's paper "Extending the Haskell Foreign Function Interface with Concurrency": http://community.haskell.org/~simonmar/papers/conc-ffi.pdf

I think that a much better alternative to Node than Python or Go is Haskell. Why? Well, for one, it makes writing asynchronous code very easy--you essentially get it for free. Haskell web servers perform very well while keeping the code readable. However, ignoring performance, there are also all of Haskell's other advantages. The type system is great--you can do stuff like ensuring links within your app work using the type system. Haskell is also very good at embedding DSLs, which can make writing web-specific code much nicer. In my experience, Haskell code tends to be more concise and declarative than Go or Python, making it easier to read. And, crucially, you can program at a higher level of abstraction which makes it both easier to write and maintain relatively complicated code. Now, admittedly, going from almost any other language to Haskell is quite a shift. For most programmers, learning Haskell will take a bit longer than learning yet another imperative language (e.g. Go). However, I think this is well worth it, and web apps are a particularly convenient field for it. This[1] StackOverflow question and an associated paper[2] are very interesting to read about how a Haskell web server could work and have good performance. [1]: http://stackoverflow.com/questions/3847108/what-is-the-haskell-respons… [2]: http://community.haskell.org/~simonmar/papers/web-server-jfp.pdf It would also be useful to look at a Haskell web framework like Yesod[3], Snap[4] or…

You can find more info about how Haskell does it here: http://community.haskell.org/~simonmar/papers/multicore-ghc.pdf

I like to tell people that the killer app for Haskell is writing IO bound, asynchronous code. The secret weapon is do-notation, which lets you write code as if it were sequential, but have it desugar into what is (essentially) a series of chained callbacks. I like to point at Facebook's use of Haskell as a good example of being successful in this space http://community.haskell.org/~simonmar/papers/haxl-icfp14.pdf It would be disingenuous to suggest that Haskell is good in all situations, but if there was one place where it should be used, this it.

> * when you look for concurrent haskell you're directed towards the corresponding paper, "tackling the awkward squad" and a world of Async, MVar, ThreadId, STM and various types of Chan, and at that point you still can't write a line of concurrent haskell I was directed here: http://community.haskell.org/~simonmar/pcph/ And after reading the first few chapters, I'd place go's options under the Rich Hickey quote of simple versus easy. Additionally I sped up a simulated annealing program with the accelerate framework in Haskell by typing "Acc " twice. From 1 000 iterations/minute to about 500 000. I'd call that an easy win. Haskell also allows for more than simply concurrent programming. I always ended up with locks in go and wondering why I was using it over c with an event loop and threads at that point.

Maybe this paper[0] on using Applicatives for performance and concurrency in Haskell's Haxl will help. Also note that there is a feature proposal[1] for Applicative Do notation. 0: http://community.haskell.org/~simonmar/papers/haxl-icfp14.pdf 1: https://ghc.haskell.org/trac/ghc/wiki/ApplicativeDo

Here's our paper about the ideas behind Haxl: http://community.haskell.org/~simonmar/papers/haxl-icfp14.pdf

This feels like lazy evaluation bolted on to Clojure (cool!). There's further benefit since you get some dataflow abstraction—which then reminds me of the Par monad. http://community.haskell.org/~simonmar/par-tutorial.pdf

Indeed, there is a pool of OS threads, a pool of Haskell lightweight threads, and a spark pool. Check out these papers: * Runtime Support for Multicore Haskell[1] * Seq no more: Better Strategies for Parallel Haskell[2] [1]: http://community.haskell.org/~simonmar/papers/multicore-ghc.pdf [2]: http://community.haskell.org/~simonmar/papers/strategies.pdf

Sigma is quite advanced; it's essentially an online DSL where authors can push anti-spam rules (written in Haskell) into live production by reloading code at runtime. Queries are done against Sigma in real time by other services, and in turn, Sigma has to query a lot of other data sources continuously in order to determine whether a rule may fire. For example, a particular rule about the nature of some of your friends on Facebook may need to query 10 different data sources (different DBs, caches, monitor infrastructure). One of the really nice things about Sigma is that it's built on Haxl, a library for efficient concurrent data access. It can also optimize the typical 'N+1 Query problem' away. What this means is you can write a program like: ids := getAllUserIds -- fetch from source 1 time foreach id as ids { x <- getUserFriends id -- N queries, 1 for each id ... } Which is simple and naive, yet Haxl can optimize this automatically into a program that will A) batch all of the data accesses together (so instead of running N queries for each ID, each query gets batched into one request for a range of users), B) automatically access each data source concurrently with no programmer intervention, so when queries can execute in parallel they do so, and C) cache the results, so that you aren't re-querying already fetched data. There's a very good paper by Simon, the author of this blog post, discussing the design of Haxl and its use:…

For more practical examples like this, I highly recommend the Haskell Data Analysis Cookbook [1]. After learning the basics of Haskell, having a book chock full of practical examples of things like handling CVS files, JSON, trees, graphs, machine learning, and Haskell's outstanding support for parallel computation--is a really helpful resource. The code is extremely well written; clear, concise, and readable. The author has also put the source code on GitHub for each chapter. [2] Afterwards, to take your skills to the next level, check out Simon Marlow's Parallel and Concurrent Programming in Haskell [3]. Simon now works at Facebook, where they're using Haskell in production [4] [5]. [1] http://www.amazon.com/Haskell-Analysis-Cookbook-Nishant-Shukla/dp/1783… [2] https://github.com/BinRoot/Haskell-Data-Analysis-Cookbook [3] http://www.amazon.com/Parallel-Concurrent-Programming-Haskell-Multithr… [4] https://code.facebook.com/posts/302060973291128/open-sourcing-haxl-a-l… [5] http://community.haskell.org/~simonmar/papers/haxl-icfp14.pdf