Server Density Blog

Personally and professionally, I have had a great experience with Datadog (https://www.datadoghq.com/) - plenty of integrations with various services, providers and a useable interface. Prior to DD, I used ServerDensity (https://www.serverdensity.com/) - also a very mature tool and the technical blog written by their founder (https://blog.serverdensity.com/) has interesting tidbits from time to time. I'm sure someone else will mention New Relic, too - great tool, but more expensive than the other two (though well worth it, if you need it)

Time series data is actually a great fit for MongoDB. You create a collection for each measurement, JSON document for each day and then preallocate the fields for each hour/minute/second. With batching you can update 100K documents a second pretty easily off one node. For example: https://blog.serverdensity.com/using-mongodb-as-a-time-series-database…

MongoDB seems to be a love/hate relation. I haven't used it myself, but I found this article[1] very useful. My conclusion is that MongoDB works, but you need to understand it and take some time to plan the deployment. [1] https://blog.serverdensity.com/does-everyone-hate-mongodb/

These "Why technology X is worthless" types of posts and statements always give me a bit of a laugh. As with most technologies, there is a) a learning curve, and b) best practices to follow. Many of the comments here seem to discuss a few things which, in my opinion, lead me to believe that the implementation is incorrect or decisions are being made on old data or based on older versions of MongoDB. Making arguments against any product based on previous versions seems to be counter productive. Most of the posts here don't make reference to a specific MongoDB version, but many reference their experience with the product in 2012. Assuming that these experiences were at the end of 2012 with the most current at the time version, we are still talking about version 2.2.2. The current stable version is 3.4.6. As you can imagine, there have been many advances to the product in five years. Basing one's knowledge and opinion of MongoDB on old versions doesn't seem logical. Even back in 2012 though, there was a lot of misinformation about MongoDB. A blog post from that time period (https://blog.serverdensity.com/does-everyone-hate-mongodb/) argues some of that information at that time. Many other comments were made based on what seems to be a lack of spending time with learning MongoDB. It is a non-relational document store. It is NOT a relational database. It requires a different way of storage design and thinking and attempting to force MongoDB to be a SQL-like database is silly.…

Performance (for any system) is really going to be dependent on your particular hardware, your database schema, and the sorts of queries you'll likely be running. Somebody else's benchmarks numbers may not mean very much for your own real-world use case. So in short - you should be running your own tests/benchmarks. Nonetheless, ServerDensity has some of their numbers here: https://blog.serverdensity.com/mongodb-benchmarks/ Also, MongoDB (the company) dogfoods MongoDB (the product) to run their online monitoring/backup service, MMS (http://mms.mongodb.com/). The scale on that thing (in terms of writes/updates) is huge. There are also several other well-known very-large installations of MongoDB: * Trello * FourSquare * Craigslist * Server Density * Trello * Bit.ly * Chartbeat Most of these DB products are reasonably mature (for a given value of mature) by now - so any of them could probably scale to your workload. So you also need to look at other features as well - developer ease-of-use, ecosystem/tooling, support etc. Put it this way - needing to scale is usually a good problem you want to be having =) - and as the above shows, MongoDB is more than capable of doing it. If you have specific questions - the MongoDB Google Group is a fantastically friendly place to ask: https://groups.google.com/forum/#!forum/mongodb-user and the answers they give are pretty detailed.

A few comments on the problems: * CPU bottleneck. The mongod is no usually bound by CPU except for building indexes on existing data (which shouldn't really happen in production). The issue he's talking about is contention between the web server (or workers) and the mongos. This isn't anything unexpected. It's recommended to put the mongos onto the application server and then you scale this by adding CPUs initially but then by adding multiple application nodes and using a load balancer. Or perhaps splitting the mongos onto dedicated nodes? * Virtualisation: VMs are notorious for having variable performance because you have the overhead of the hypervisor but more importantly, are sharing resources with others. We run performance critical apps on dedicated servers and reserve VMs for tools or things which aren't high throughput e.g. a MongoDB arbiter. * EBS: This has had known performance issues for years. It's fine as a basic file store but should never be used for databases. PIOPs are the way around this but local instance storage is also an option. * No transactions: MongoDB has never had them. This is known. * Schemaless != no schema design. It makes it easy to play around but you still need to think through things carefully. See http://blog.serverdensity.com/mongodb-schema-design-pitfalls/ * No joins. Again, it has never had joins. This is known.

Same reaction here - this is based on the premise that the data was transferred directly from the DNC server to somebody's home connection, and doesn't allow for the possibility that it was first transferred to some server in a data center. https://blog.serverdensity.com/network-performance-aws-google-rackspac… Just some random article from 2014, look at the data points for transfers between data centers where anybody can rent servers: Amazon "us-east" <-> "us-west" 210 mbit/s Google "us-central" <-> "europe-west" 189 mbit/s Rackspace "Dallas" <-> "North Virginia" 534 mbit/s Softlayer "Dallas" <-> "Amsterdam" 61 mbit/s

I didn't meant external networking. Connect two datacenters, see what costs come up. Connect machines inside your datacenter, etc.. Your maintenance cost will raise. AWS is cheap. Whatever you did, you operated on a really slow scale where money didn't mattered. Or you completely forgot a lot in your calculation. Oh and here is a Link from Zynga: http://readwrite.com/2015/05/12/zynga-data-center-amazon-aws-public-cl… My next examples would be Gilt, etc. You can't run elastic in your own datacenter. From your article: > https://blog.serverdensity.com/saving-500k-per-month-buying-your-own-h… > Dell R415 with x2 processors each with 8 cores, 32GB RAM, a 500GB SATA system drive and a 400GB SSD This one made me laugh...

I do Devops/Infrastructure for a living, I've shown the math to both my day gig and numerous consulting clients that AWS is 30-40% more expensive, even when taking into account reserved instance pricing. External networking expecially is super expensive compared to traditional transit pricing. They're absolutely wonderful for bursty or async workloads, but if you know your load, you're better off on your own gear. EDIT: Source added. https://blog.serverdensity.com/saving-500k-per-month-buying-your-own-h… "Saving $500k per month buying your own hardware – cloud vs colocation" The article mentions SoftLayer, but its just as relevant for AWS.

https://blog.serverdensity.com/does-everyone-hate-mongodb/ They use it as well for similar purposes. So it isn't a strange choice. That said, I'd use Cassandra as well. ;) I found when testing the write performance was significantly better. However, I didn't test the special Mongo variant they are using.

Cost: https://blog.serverdensity.com/saving-500k-per-month-buying-your-own-h…

Because some people don't use it correctly doesn't mean it can't be done. Here is a different random article about ServerDensity using it successfully https://blog.serverdensity.com/tech-behind-time-series-graphs-2bn-docs… This is nothing wrong with MongoDB.

I thought global lock is gone in v2.2 https://blog.serverdensity.com/goodbye-global-lock-mongodb-2-0-vs-2-2/

The global lock was removed in 2.2 https://blog.serverdensity.com/goodbye-global-lock-mongodb-2-0-vs-2-2/ Now locking is on the database level.

Proof that the OP was just looking for more Hacker News cred so they wrote about why MongoDB sucks but in reality wanted to discuss their "human error" :p Nice marketing guys http://blog.serverdensity.com/does-everyone-hate-mongodb/

heh, I thought the domain sounded familiar: http://blog.serverdensity.com/removing-memcached-because-its-too-slow/

Actually, there is a lot of difference but not just because of the global lock to database lock transition. There is also a lot of work done on yield-before-lock: http://blog.serverdensity.com/goodbye-global-lock-mongodb-2-0-vs-2-2/ And don't forget that replication also happens in a different database...

A timeline of Google's messaging rivalry with Facebook. Original inspiration from source [23], enhanced and corrected from [33]: - 2006-02-07: Google Talk integration inside Gmail goes live [35] - 2006-09-26: Facebook opens up to everyone (not just colleges) [36] - 2007-02-14: (corrected date) Gmail opens up to everyone (not just invite-only) [37] - 2008-04-06: Facebook chat goes live [34] - 2008-07-11: iOS App Store launches [38] - 2008-08-26: Facebook hits 100 million active users [24] - 2008-09-23: Android 1.0 launches [39] - 2008-11-11: Google Talk introduces voice and video calling [40] - 2009-03-11: Google buys GrandCentral, launches Google Voice [1] - 2009-04-08: Facebook hits 200 million active users [25] - 2009-06: iOS gets push notifications [42] - 2009-06-25: Google Voice invitations being serviced [2] - 2009-09-15: Facebook hits 300 million active users [26] - 2010-02-04: Facebook hits 400 million active users [27] - 2010-05-20: Android gets push notifications [41] - 2010-06-21: FaceTime released with iOS 4 [43] - 2010-06-22: Google Voice opens up invite-free to everyone in the US [3] - 2010-07-21: Facebook hits 500 million active users [28] - 2011-01-05: Facebook hits 600 million active users [29] - 2011-05-30: Facebook probably hits around 700 million active users [30] - 2011-06-28: Google+ launches, with text chat "+Messenger" and video chat "+Hangouts" [4] - 2011-07-06: Facebook introduces video calling powered by Skype…

Half our industry is built on fooling users. Exploiting their cognitive biases. Seriously, we understand where they fall, and if we really wanted to, we absolutely could look out for them -- at least, certainly better than we're doing right now. Your response is like saying when Facebook was privacy zuckering (http://darkpatterns.org/library/privacy_zuckering/), the user clicks right through the settings that should have sounded an alarm. What can Apple do? They could make a better browser (I like how Chrome and Firefox do things - you have to go out of your way to reach a page with bad SSL -- compare Safari's rather passive and enabling error message: http://blog.serverdensity.com/wp-content/uploads/2009/05/ssl-error.png vs. chrome's: http://i.imgur.com/ttmmDJ8.png -- you have to REALLY see and think how to access the site despite the warning, it's that good). They could be more vigilant in alerting users of where and how this can happen. If we were talking about any other young startup, your apology might fly -- not so with Apple, they're sitting on billions, they have the resources to think of a solution and implement it.