M
MongoDB Blog
blog.mongodb.org · Company blog · on HN since 2009
Official MongoDB company blog covering database features, durability, distributed consistency, and customer deployments.
What HN saysHN commenters cite it for MongoDB's own statements on durability, 32-bit limits, and journaling, and for how-to material on mixing SQL and NoSQL.
90HN people shared it
33linked it in comments
100stories submitted
1.8Kstory points
10 years agolast shared
2009people per year2026
Most-shared posts
- 1Why MongoDB thinks single server durability is overvalued6 people · 49 points · first shared 2010 · 29 comments on HN
- 232 bit limitations6 people · first shared 2009
- 3The MEAN Stack: MongoDB, ExpressJS, AngularJS and Node.js4 people · 92 points · first shared 2013 · 81 comments on HN
- 4MongoDB live at Craigslist - an interview with Jeremy Zawodny3 people · 13 points · first shared 2011 · discussion on HN
- 5MongoDB: Operations in the New Aggregation Framework3 people · 9 points · first shared 2012 · discussion on HN
- 6On Distributed Consistency (6 parts)3 people · 4 points · first shared 2010 · discussion on HN
- 7Introducing Mongo Connector3 people · 1 points · first shared 2012 · discussion on HN
- 8The agpl3 people · first shared 2010
- 9MongoDB 3.0 Released2 people · 16 points · first shared 2015 · discussion on HN
- 10Introducing the MongoDB Driver for Rust2 people · 15 points · first shared 2013 · 1 comments on HN
- 11Using Go for MongoDB Backups2 people · 8 points · first shared 2013 · discussion on HN
- 12How MongoDB’s Journaling Works2 people · 3 points · first shared 2012 · discussion on HN
- 13Transitioning from Relational Databases to MongoDB - Data Models2 people · 2 points · first shared 2014 · discussion on HN
- 14MongoDB fantastic for logging2 people · 1 points · first shared 2009 · discussion on HN
- 15MongoDB 2.0 Released1 people · 215 points · first shared 2011 · 76 comments on HN
What HN people wrote when they linked it
thom_nic · 2016-08-01
Yes, apparently the limit on 32-bit is 2GB actually. MongoDB has always stated upfront that 32-bit architectures are not recommended for production use for precisely this reason: http://blog.mongodb.org/post/137788967/32-bit-limitations
This has also has been stated on their download page for 32-bit binaries as well.
niftich · 2016-07-25
Mongo had better marketing. They sponsored hackathons, they had great out-of-the-box UX, they filled their homepage with logos of both 'big enterprise' and 'hip startup' companies using it, and were able to present their product in a way to capitalize on the rise of Node.js and the 'NoSQL' keyword hype.
In fact, the 'MEAN stack' was coined by a MongoDB developer on his blog [1], a post which is a perfect example of the kind of marketing, anecdotes, messaging, and strategies that Mongo employed to stay relevant.
[1] http://blog.mongodb.org/post/49262866911/the-mean-stack-mongodb-expres…
jonaldomo · 2015-03-10
I couldn't agree more with you. I love SQL for when I have a concrete data model, when I don't I love NoSQL. I could use both if I really wanted to, but you need to know when to force a square peg into a circle. Mongo has a good how to on it here. http://blog.mongodb.org/post/72874267152/transitioning-from-relational…
arkham · 2014-04-08
Just to note, the MongoDB journal is basically a write ahead log and has been around since 1.8. It's not simple, you are correct, and involved remapping portions of memory privately, and leads to some inflated numbers on the virtual memory reporting side. There's a great write up here:
http://blog.mongodb.org/post/33700094220/how-mongodbs-journaling-works
rdtsc · 2011-11-01
> caught acknowledging
instant classic.
Those that can, comment, those that have nothing to say, make fun of those who speak English as a third language.
http://blog.mongodb.org/post/381927266/what-about-durability
Feb, 2010
"We get lots of questions about why MongoDB doesn’t have full single server durability, ...there are some very practical reasons why we think single server durability is overvalued"
http://www.mongodb.org/pages/viewpage.action?pageId=2752708&naviga…
May, 2010
"MongoDB does not publish any official benchmarks...."
EDIT: Added link to blogs, quoted parent, in case gets removed
simonw · 2009-08-27
They wrote about MongoDB for logging recently: http://blog.mongodb.org/post/172254834/mongodb-is-fantastic-for-loggin… - capped collections sound ideal for short lived logs.
It would be interesting to write a command line script for piping Apache logs in to MongoDB, using Apache's "piped logs" feature: http://httpd.apache.org/docs/2.0/logs.html#piped
pixelmonkey · 2012-04-14
This is a pretty epic troll on MongoDB, and some of their points are important -- particularly global write lock and uncompressed field names, both issues that needlessly afflict large MongoDB clusters and will likely be fixed eventually.
However, it's pretty clear from this post that they were not using MongoDB in the best way. For example, in a small part of their criticism of "safe off by default", they write:
"We lost a sizable amount of data at Kiip for some time before realizing what was happening and using safe saves where they made sense (user accounts, billing, etc.)."
You shouldn't be storing user accounts and billing information in MongoDB. Perhaps MongoDB's marketing made you believe you should store everything in MongoDB, but you should know better.
In addition to that data being highly relational, it also requires the transactional semantics present in mature relational databases. When I read "user accounts, billing" here, I cringed.
Things that it makes total sense to use MongoDB for:
- analytics systems: where server write thorughput, client-side async (unsafe) upserts/inserts, and the atomic $inc operator become very valuable tools.
http://blog.mongodb.org/post/171353301/using-mongodb-for-real-time-ana…
- content management systems: where schema-free design, avoidance of joins, its query language, and support for arbitrary metadata become an excellent set of tradeoffs vs. tabular storage in an…
jraedisch · 2015-01-14
Could you provide a source for "multi document transactions" support in 2.8? According to this comment they are supported by WiredTiger but won't be in the MongoDB API anytime soon: http://blog.mongodb.org/post/102461818738/announcing-mongodb-2-8-0-rc0…
twunde · 2014-12-20
I second the idea of denormalizing your data. It feels odd because that's exactly what you're trying to avoid with relational databases, but honestly, mongo is designed to handle denormalized data
http://blog.mongodb.org/post/88473035333/6-rules-of-thumb-for-mongodb-…
calpaterson · 2014-09-23
Some updates are done in place, some are not. To give a specific example - integer updates are always done in place. The most common reason that a document has to be rewritten is when it is updated size is too big to fit into its previous space. Then it has to be moved.
http://blog.mongodb.org/post/248614779/fast-updates-with-mongodb-updat…
I have found "source diving" a scary experience with Mongodb in the past.
yeukhon · 2014-07-27
The APGL license caught my eyes. Does anyone take this license into account (deciding between an Apache/MIT/BSD DB vs APGL DB) when they use it in their service / software stack.
For example, this openstack thread always keeps me alert using APGL when I am developing a solution.
http://lists.openstack.org/pipermail/openstack-dev/2014-March/030510.h…
and here is MongoDB's FAQ explaining APGL in plain English: http://blog.mongodb.org/post/103832439/the-agpl
spf13 · 2014-05-16
I work for MongoDB. There we use go for all of our agents for our various services. Originally they were developed in java and python, but we spent far too much time supporting issues that arose from existing environments run times. A blog post was written that expands on this further. http://blog.mongodb.org/post/51643994762/go-agent-go
In my free time I develop a static site engine called Hugo which is hundreds of times faster than others written in python and ruby and is super easy to install as it is distributed as a single binary without any dependencies. You can learn more about it at http://hugo.spf13.com .
citruspi · 2014-04-08
I believe the 2 GB database limit is only applicable to 32-bit instances[0].
[0]: http://blog.mongodb.org/post/137788967/32-bit-limitations
taude · 2013-10-31
The Mean Stack: MongoDB, Express JS, AngularJS, and NodeJS [1]
[1] http://blog.mongodb.org/post/49262866911/the-mean-stack-mongodb-expres…
ranman · 2013-08-27
Also an amusing blog post from the CTO:
http://blog.mongodb.org/post/59489943730/todays-news
stesch · 2013-08-03
Rust is a perfect example: http://blog.mongodb.org/post/56426792420/introducing-the-mongodb-drive…
So are many other newer languages and frameworks. See vibe.d (http://vibed.org/) for D which has only database drivers for MongoDB and Redis.
tsenkov · 2013-07-24
To kickstart a discussion:
Does anyone use the MEAN stack: mongoDB, ExpressJS, AngularJS and NodeJS?
http://blog.mongodb.org/post/49262866911/the-mean-stack-mongodb-expres…
I am starting a project and currently I am leaning towards NodeJS with Express and MongoDB on the server-side and RequireJS(or WireJS) and Knockout on the client side.
I intend to use QUnit for the client app's testing, but have no idea what should I use for Node testing (unit and load/stress).
Since it will be just me, I will probably setup Jenkins on my own machine for a build server.
I am currently using appfog for staging/testing.
That's pretty much it. (My project includes a mobile part as well, but I guess this is outside of the scope of this discussion, so I wont get into that.)
benmccann · 2012-12-06
I really wish they had been more specific about the problems they faced. It sounds like part of the headache was that they had to write their own MongoDB to ElasticSearch connector. However, 10gen has now written one which they could use to avoid writing and maintaining their own:
http://blog.mongodb.org/post/29127828146/introducing-mongo-connector
meghan · 2012-08-29
Blog post with the details is up http://blog.mongodb.org/post/30451575525/mongodb-2-2-released
taligent · 2012-08-13
There is this:
http://api.mongodb.org/hadoop/MongoDB%2BHadoop+Connector.html
And a generic connector as well:
http://blog.mongodb.org/post/29127828146/introducing-mongo-connector
Ecio78 · 2011-10-31
afaik single server durability was a problem in the past. Since 1.8 they've introduced journaling and it's now enabled by default in 2.0
More info here: http://blog.mongodb.org/post/381927266/what-about-durability and here: http://www.mongodb.org/display/DOCS/Durability+and+Repair
jwilliams · 2011-08-13
Keep in mind this is a year old and MongoDB has since implemented journalling (although afaik it's not on by default yet).
It's not specifically a response to this article - But you can read 10gen's view on durability here:
http://blog.mongodb.org/post/381927266/what-about-durability
skorgu · 2011-06-16
That's how the MongoDB guys interpret it[1]:
To say this another way: if you modify the core database source code, the goal is that you have to contribute those modifications back to the community.
Note however that it is NOT required that applications using mongo be published.
[1] http://blog.mongodb.org/post/103832439/the-agpl
zefhous · 2011-05-17
The link should probably be to this page:
http://blog.mongodb.org/post/5545198613/mongodb-live-at-craigslist
kchodorow · 2011-05-11
MongoDB is partition tolerant and consistent.
You can never have multi-master with MongoDB, which is required for "always writable." However, it can be readable. Our CEO did a series of posts on distributed consistency, see http://blog.mongodb.org/post/475279604/on-distributed-consistency-part…
Semiapies · 2010-09-09
4) You're using a 64-bit OS. http://blog.mongodb.org/post/137788967/32-bit-limitations
acangiano · 2010-06-12
Straight from the horse's mouth (http://blog.mongodb.org/post/103832439/the-agpl):
"To say this another way: if you modify the core database source code, the goal is that you have to contribute those modifications back to the community.
Note however that it is NOT required that applications using mongo be published. The copyleft applies only to the mongod and mongos database programs. This is why Mongo DB drivers are all licensed under an Apache license. You application, even though it talks to the database, is a separate program and “work”."
dm_mongodb · 2009-12-31
this touches on some related aspects: http://blog.mongodb.org/post/307919034/partial-object-updates-will-be-…
herf · 2009-08-10
MongoDB appears to memory map its data: http://blog.mongodb.org/post/137788967/32-bit-limitations
This is fine on 64-bit CPUs, but less good on 32-bit dev boxes that people are probably using.
Very nice API, though.
cheald · 2015-07-20
> All the wrong places, as already stated. These are not the places where developers actually look.
If you're not reading READMEs, installation notes, and server logs and instead depend on marketing splash pages to highlight the limitations of the technologies you use, you're a bad developer. Sorry.
> Certainly not [in blog posts]
http://blog.mongodb.org/post/137788967/32-bit-limitations
> It simply wasn't in the locations where people actually look. You can easily verify that using the Wayback Machine (and in fact, this is still mostly the case today).
I first started using Mongo in 2010 according to my gitlogs (after it having been on my radar for some time before that), and I very distinctly remember reading about the 32-bit limitation well before I'd ever installed it, because we were also using Varnish at the time, which has similar issues on 32-bit systems because it too uses (or at least, at that time used) memory-mapped files.
The claim that the limitation was not documented well, or was hidden, or was generally unknown at that time is just simply false.