Bret Taylor

best thing to come out of friendfeed, Tornado. I remember asking Bret Taylor on his blog [0] when he gave a Tornado example if he'd ever release it. A bit of time went by, then he did. [1] Thanks Bret. [0] http://backchannel.org/blog/tornado [1] https://friendfeed.com/friendfeed-news/9b8fbaed/technology-behind-torn…

Not in 2015/2016 when so many choices were already available, and certainly not in an org where MySQL would just be more than enough to meet Uber's need. Besides, Uber built their version on top of MySQL following pretty much FriendFeed's design: https://backchannel.org/blog/friendfeed-schemaless-mysql. A strange choice by itself given that better solutions had popped up left and right during the 6 years since the blog was published. Uber switch to Schemaless from a Postgres cluster, claiming that Postgres "does not scale for Uber". I see irony in that piece of Uber history. Particularly, Uber always claimed that some OS solution "had serious reliability issues". Cassandra in 2014 worked for companies that had orders of magnitude of traffic, yet Uber found it unusable. Similarly, Uber switched from MySQL to Postgres and then from Postgres to MySQL. Each time Uber published an article to claim engineering victory. The truth is that any of the three systems would be perfectly fine for Uber. Oh, didn't someone also claim that Elaticsearch crashed under Uber's load" and JVM-based systems were not usable for Uber's scale? " before Uber ended up adopting Elasticsearch? What really happened was that some teams in Uber simply didn't want to dive deep and produce objective tech assessment.

One of the post's points is that UUIDs will scatter your writes across the database, and that for this reason you want a (more or less) sequential key as your primary key. This crucially depends on both your database technology and your query patterns. In a single-node database or even a manually-sharded one, this post's advice is good (For Friendfeed, we used a variation of the "Integers Internal, UUIDs External" strategy on sharded mysql: https://backchannel.org/blog/friendfeed-schemaless-mysql). But in a distributed database like CockroachDB (Disclosure: I'm the co-founder and CTO of Cockroach Labs) or Google Cloud Spanner, it's usually better to get the random scattering of a UUID primary key, because that spreads the workload across all the nodes in the cluster. Sometimes query patterns benefit enough from an ordered PK to overcome this advantage, but usually it's better to use randomly-distributed PKs by default. For CockroachDB, my general recommendation for schema design would be to use UUIDs as the primary keys of tables that make up the top level of an interleaved table hierarchy, and SERIAL keys for tables that are interleaved into another. (Google's recommendations for Spanner are similar: https://cloud.google.com/spanner/docs/schema-design#choosing_a_primary…

Can someone explain to my why people are still recommending mysql over postgres? This is a serious question, it just seems that Postgres has more features and I cant think of any good reasons mysql would scale any differently other than it has been along a little longer (so there are more blog posts + experienced engineers) ? "MySQL is mature, stable and it just works. Not only do we use it, but it’s also used by plenty of other companies pushing even bigger scale. MySQL supports our need for ordering data requests, selecting certain ranges of data and row-level transactions. It has a hell of a lot more features, but we don’t need or use them. But, MySQL is a single box solution, hence the need to shard our data. Here’s our solution:" what about that paragraph is not true of postgres also ? UPDATE: This schemaless json reminds me of this friendfeed blog post from years ago: https://backchannel.org/blog/friendfeed-schemaless-mysql

This might be of help: https://backchannel.org/blog/friendfeed-schemaless-mysql

Thanks for your support. The data is stored in a sqlite3 database. The next two big features will be network awareness (that is, you can have multiple databases sync with eachothers) and custom data fields where I was planning doing something like http://backchannel.org/blog/friendfeed-schemaless-mysql to get a fully customable configurations.

From 7 years ago, by Bret Taylor (who went on to become CTO at Facebook after acquisition): How FriendFeed uses MySQL to store schema-less data https://backchannel.org/blog/friendfeed-schemaless-mysql Edit to add the HN discussion at the time: https://news.ycombinator.com/item?id=496946

related: friendfeed used a similar approach https://backchannel.org/blog/friendfeed-schemaless-mysql

This was done by FriendFeed back in 2009: https://backchannel.org/blog/friendfeed-schemaless-mysql

Reminds me of: http://backchannel.org/blog/friendfeed-schemaless-mysql

This reminds me of how FriendFeed was using MySQL to store unstructured data. http://backchannel.org/blog/friendfeed-schemaless-mysql

This reminds me of FriendFeed´s Schemaless MySQL. https://backchannel.org/blog/friendfeed-schemaless-mysql They were acquired by FB.

If you watched the talk, he explains the specific kinds of that should be stuffed into a JSON text column. It makes a difference. This practice been done by many production sites over the years http://backchannel.org/blog/friendfeed-schemaless-mysql Might not be considered bad practice now.

If you really wanted, you could always take a FriendFeed style approach and create a document store on top of SQLite. http://backchannel.org/blog/friendfeed-schemaless-mysql

This is similar to what FriendFeed did in 2009 - storing schema less data into MySQL: http://backchannel.org/blog/friendfeed-schemaless-mysql

FriendFeed actually used a similar mechanism with MySQL for some schemaless data that they knew they wouldn't need to index. "Entity bodies are stored as zlib-compressed, pickled Python dictionaries." http://backchannel.org/blog/friendfeed-schemaless-mysql

What if it is the difference between your schema working under high transaction rates or not? Or, the difference between buying much more database hardware and having smaller shards to avoid index contention. (which, is $$) Facebooks mysql architecture (at least used to, likely still is) based on this: https://backchannel.org/blog/friendfeed-schemaless-mysql Note the simple schema, and lack of FKs. FKs are rarely present in extremely high transactions per second systems that operate on the same few tables. This is due to index contention and locking. I say this with lots of experience working on different production systems that see hundreds of thousands of transactions per second. I have yet to see one arrive at these kind of numbers using foreign keys, unless it's something like a giant shared hosting platform, that is operating on hundreds and hundreds of different tables. (therefore, less index contention) Often, in these shops, data correctness is not validated in realtime in this way, but often in a way that is not in the critical path of answering queries. (more like, eventually consistent). Some places have entire teams for this. To your point on correctness, my experience ranges a lot in areas where correctness can be eventually consistent. For something like healthcare or banking, you would rather spend the money on way more hardware, because you can't afford correctness to be off. However, with things like globally scaled social apps, this is just not…

Should be pretty trivial for him to shard collections across multiple databases with the same level of intelligence as MongoDB's automated sharding simply using the primary key, since MongoDB doesn't join. Not sure how sharding is the point of MongoDB, though - in most of the universe, sharding is a database architecture/schema-level thing, not a database-server level thing, and for good reason - it's pretty darn hard for a database server to shard effectively without some knowledge of the app layer (and which keys are likely to become hot). Personally, if I really wanted a flexible-schema "document based" database, I'd have implemented this using the FriendFeed K/V + Index model ( http://backchannel.org/blog/friendfeed-schemaless-mysql ) plus Postgres's HStore functionality, storing K/V per document in an HStore rather than in one giant K/V table like FriendFeed. That way I wouldn't need to use V8 and JSON parsing to run queries, and the mythic MongoDB-style "sharding" would be just as easy (just distribute the document -> hstore table across shards keyed on ID again).