Basho Blog

I've found the resources the folks at Basho (makers of Riak) have put together on their wiki and blog quite useful. The following would be a good starting point: http://blog.basho.com/2010/03/19/schema-design-in-riak---introduction/ http://blog.basho.com/2010/03/25/schema-design-in-riak---relationships… http://wiki.basho.com/ http://blog.basho.com/

Here's the actual slides to my talk: http://www.slideshare.net/argv0/distributed-erlang-systems-in-operatio… The video should be up sometime soon on http://erlang-factory.com, or you can follow me at http://twitter.com/argv0 and I'll be sure to link to the video of the talk when it comes out. There's also a lot of similarly interesting content on http://blog.basho.com

Dynamo seems to be still pretty early in development. You might want to watch the video linked to the page from the 0redev conference. He doesn't talk about dynamo much, but he does give "2013" as the date for Dynamo. On the WebSockets issue, I'd check out Cowboy. Cowboy seems to be the new hotness in erlang web apps. It has at least some support for web sockets. https://github.com/extend/cowboy If you're looking for a full featured web framework with built-in web socket support then maybe what you really want is Nitrogen: http://nitrogenproject.com Finally, and just for completeness, since I'm mentioning web frameworks, you'll probably want to see what ChicagoBoss's WS story is: http://www.chicagoboss.org or if someone's done something with WebMachine and websockets: http://basho.com I think Nitrogen sounds like the best bet. Since Elixir seems to mix effortlessly with Erlang, you should be able to write your code in Elixir using any of these frameworks.

Take a look at Riak, http://basho.com/. Disclosure, I have recently joined up with them as a Solutions Architect.

Interesting, I have seen CRDTs mostly in the Erlang context (monitoring and following works Basho has been doing for Riak). Basically a lot of this: http://basho.com/tag/crdt But using Haskell's type system is an interesting idea. You can start with one level higher than just looking for associative, commutative and idempotent, because that actually comes the definition of a semilattice http://en.wikipedia.org/wiki/Semilattice The crucial part is that you have an appropriate meet or join operator (and that it satisfies the above requirements). Informally think of a functions like max() over natural numbers or a union() over sets. Those are some examples. I don't know Haskell but found this interesting gist of someone who has tried this, and frankly a lot of stuff there is above my head, but it might help you: https://gist.github.com/pozorvlak/6540639

From firsthand experience, the PostGIS extension for PostgreSQL is fantastic. A couple of my coworkers at Basho have done geospatial work with Riak, our scalable, distributed database: http://basho.com/indexing-the-zombie-apocalypse-with-riak/

Great to hear that you're cutting over to Riak. We highly recommend that your minimum cluster size is your N value (replication value) + 2. I your case that likely means 5 nodes for your starting cluster, not 3. There are many reasons. http://basho.com/blog/technical/2012/04/27/Why-Your-Riak-Cluster-Shoul…

You have to _think_ about conflict resolution, but it doesn't have to be hard. If you're a developer who needs to use Riak this talk[1] is priceless. The riak-java-client[2] provides a simple way to hook conflict resolution into your code (thanks to the ideas and solutions in that talk.) There is also the excellent Statebox from Mochi, which automates conflict resolution. [1] Riak and Scala at Yammer - http://blog.basho.com/2011/03/28/Riak-and-Scala-at-Yammer/ [2] Riak-java-client - https://github.com/basho/riak-java-client/ [3] Statebox - https://github.com/mochi/statebox

This looks like a really cool project/language. How do I build it? Does it build on OS X? -- I'd love to "watch" the project on github, but alas it's stranded on the island of google code. Check out this [1] basho blog entry on why they chose github over bitbucket — most of the reasoning applies to google code as well. I'm not affiliated with github, by the way, just a happy user. For a project like this, the concept of "watchers" isn't a vanity metric. I've been working on a lisp->javascript language and have found github watchers to be surprisingly motivating and encouraging. [1] http://blog.basho.com/2010/11/11/a-few-more-details-on-why-we-switched…

I would also like to see an open source javascript web app written "the right way", I have a lot of javascript code, its getting nicer, but I still feel like I am missing something. most large open source javascript is libraries and such, they are very different from applications for erlang it seems anything that comes out of http://www.basho.com/ is a good showcase, mochi as well.

Given their love and respect of CAP, it seems quite awesome that Dr. Brewer recently joined Basho's board of directors: http://basho.com/Eric-Brewer-Joins-Board-of-Directors-of-Basho.html

> Another such area of work is logical time, manifest as vector clocks, version vectors, and other ways of abstracting over the ordering of events. This idea generally acknowledges the inability to assume synchronized clocks and builds notions of ordering for a world in which clocks are entirely unreliable. Hardware is unreliable. Software is possibly less reliable. We have known that for a long time. The author talks on a conceptual level about logical time, but this concept isn't enough to understand the real challenges & possible solutions of keeping interactions in your system logically ordered in the dimension of time[0]. > You can think of coordination as providing a logical surrogate for "now." When used in that way, however, these protocols have a cost, resulting from something they all fundamentally have in common: constant communication. For example, if you coordinate an ordering for all of the things that happen in your distributed system, then at best you are able to provide a response latency no less than the round-trip time (two sequential message deliveries) inside that system. Consensus protocols don't provide a logical surrogate for 'now', a log does that. The silver bullet for assuring that your transactions are ordered correctly is immutability[1]--"If two identical, deterministic processes begin in the same state and get the same inputs in the same order, they will produce the same output and end in the same state.[0]" It's important, from the…

San Francisco, CA - Yammer Yammer is a cool tech startup masquerading as an enterprise software vendor. We're building an enterprise social network (think Facebook, but for your company. Also, better.) and we need your help. We build our product with insight and wisdom gained from the consumer social networking space, but we charge enterprise prices (and our customers pay them!) We're fighting some big serious competitors (Salesforce/Chatter, Jive, and VMWare/Socialcast) and this space has never been more exciting. Yammer is really changing the way people get work done. We have real challenges to overcome and we're doing our best to make a kick-ass product that makes our users happy. Tech we use: Ruby/Rails, Scala, and Node/JS. We have Obj-C and C# stuff too. Some bullet points for you to skim: • Amazing group of smart engineers to work with. Really. • We hack in Ruby/Rails, Scala/Java, Javascript/JQuery/Node.js/Adobe AIR, Obj-C for iOS, and some MSFT/Sharepoint stuff. • Competitive compensation. Enough said. • Delicious catered lunch and dinner daily, with a 3PM snack cart. (Really.) Also a fully-stocked beer-and-beverage fridge. • Fancy Apple hardware of your choice (you can have a PC if you really want one.) Some links for you to read more: • Our jobs page: https://www.yammer.com/jobs • Our Engineering blog: http://eng.yammer.com/ • A video of a talk given by Coda Hale and Ryan Kennedy about how we use Riak at Yammer:…

Without going too much into specifics and picking on individual databases (which I could do, boy do I have the scars...) When you hit a certain traffic level, scalability, latency and robustness become far more important than single-node ops/s. I need to be able to add nodes and repair failed nodes while under load--I need the 99.9% latency mark to stay ~100ms while doing so. I don't really care how many bajillions of ops a second your database can do in some concocted scenario, b/c you're not going to do that many in the real world anyway (trust me, we tried). The disk subsystem is going to give you a few hundred, maybe a few thousand if you're lucky, IOPS, then your latency will spike to hell and your phone will wake you up at night. Maybe in the world where 99% of ops are reads, you will put up impressive numbers, but now you're just showing you are pretty good at using the disk cache. That's a relatively easy problem. The riak guys seem to get all this better than most: http://blog.basho.com/2011/05/11/Lies-Damn-Lies-And-NoSQL/ So, to give you a short answer to your direct question: Use SLC SSDs + md + RAID-0. Have at least 5 nodes. Use bitcask, but realize that your keys will need to fit in memory. Also, realize that really small values aren't a great fit for Riak in some ways b/c the overhead per value is at least a few hundred bytes. Also, it's important to note this is where I'm at right now, but maybe not where you (generally) are at. Riak may not make…

Basho http://basho.com Engineers Sales/Marketing

More importantly, they acquired Russel Brown ( http://basho.com/posts/author/russell-brown/ )

Here's a relevant post from Basho about that and their take on it. http://basho.com/posts/technical/erlang-at-basho-five-years-later/

My pleasure! http://basho.com/products/ http://highscalability.com/blog/2014/2/26/the-whatsapp-architecture-fa…

Riot Games is a pretty big one. Looking at erlangs site selecting a few: AdRoll Facebook Chat TMobile http://erlang.org/faq/introduction.html Amazon SimpleDB Delicious From: http://stackoverflow.com/questions/1636455/where-is-erlang-used-and-wh… Also many of the companies using Riak (written in Erlang) also leverage erlang. Some pretty big names use it. http://basho.com/about/customers/ I would say that Erlang has more than proved itself. One aspect of it is that it is used more of an infrastructure language so not many people see it as glamorous as rushing out to the Go community to brag about their new backend. Its a lot of mature companies that have been around a while and don't really need to show off their infrastructure. Also Elixir is gaining momentum (but still a blip) as a very viable language built on erlang. Use what you want and what works.

Also to be taken with its sister post: http://basho.com/why-vector-clocks-are-hard/

If anyone from basho is here, the link[0] at the end of this article leads to a 404. I guess it might've been moved to this[1]. [0]: http://basho.com/technology/why-use-riak/ [1]: http://docs.basho.com/riak/latest/theory/why-riak/

New CEO and CTO a year ago are a likely source of repositioning: http://basho.com/basho-names-new-ceo-and-cto/ I can't speculate on how that all relates to brain drain.

For anyone else that's confused: What they're talking about: http://docs.basho.com/riak/latest/theory/why-riak/ Their github: https://github.com/basho/riak CRDT = "Conflict-free Replicated Data Type" or "Convergent Replicated Data Type" or "Commutative Replicated Data Type" This page explains what the heck CRDT is: http://basho.com/tag/crdt/

Riak[0] could also potentially be a good fit for your use case. Unlike Redis, Riak is distributed. I've heard of several companies in the POS space using it. 0 - http://basho.com/riak/

There are some fascinating stories on using QuickCheck to test Riak and other distributed systems. For example: http://basho.com/tag/quickcheck/ and https://skillsmatter.com/skillscasts/4505-quickchecking-riak (registration required)

A counterpoint: http://basho.com/erlang-at-basho-five-years-later/ Basho uses Erlang, despite not really being able to hire erlang devs (just need to hire and train, for the most part, its a small community), because they feel it gives them a competitive advantage, and even attracts more talented developers. I'm not arguing that you can't or shouldn't get shit done w/ node.js/Go/PHP/Java/C#/whatever works, just that its ignorant to say "Haskell/Lisp/Erlang/OCaml/Whatever has lots of weird shit I don't need". It's funny that you call these languages the "language de jour" when Haskell, Lisp, and Erlang are all like 20+ years old.

I see now - interesting. Big shift from 2010 to now. Thanks! http://www.slideshare.net/rklophaus/riak-search-erlang-factory-london-… http://basho.com/ricon-west-videos-riak-search-2-0/

Yeah, my first two thoughts were discodb and bitcask http://basho.com/hello-bitcask/ too.

Very smart move. Some time ago Riak did the same thing. Their rationale was more detailed, but came to the same conclusion: http://basho.com/a-few-more-details-on-why-we-switched-to-github/