Cloudant Blog

update just found a great article on scaling with CouchDB using Lounge http://nosql.mypopescu.com/post/683838234/scaling-couchdb and a company that provides the service Cloudant: https://cloudant.com/ Here's another shard manager project, Pillow: http://knuthellan.com/2010/07/14/feature-prioritization-for-pillow-the… I think the design philosophy for couch is keeping smaller pieces of Big Data synchronized. The sum of all those small pieces is like one larger database.

I just wish someone would offer a decent hosted NoSQL platform that I can start using already. https://cloudant.com/ is invite only. So is http://hosting.couch.io/ apparently. https://app.mongohq.com/signup is overpriced, considering it's a cloud service and only 2GB. SimpleDB isn't bad but it has tons of limitations (can't even sort by numbers). This is why people still use MySQL even for projects that aren't suitable for RDBMS. I use hosted MySQL at dreamhost and don't have to bother with anything except my app and data. It just works and is free with the web hosting package. Is there anything out there that comes close? I don't mind $1/month for 1GB of data. $25 for 2GB is not worth it.

I'm going to make a couple of assumptions and if I'm wrong just ignore the answer. I'm assuming you are using either PHP, Python, Java, Ruby, or .Net. I'm assuming these are either web servers, worker servers, caching servers, or database servers. With those assumptions made I'm going to suggest some services to use. I don't work for ANY of these companies. I run a small startup that uses most of these companies. For web servers I would recommend using AWS' Elastic Beanstalk. You don't pay anything extra to use this service in addition to the EC2 servers you are using now. If you use reserved instances it is also cheaper than pretty much any other PaaS out there (Google App Engine gives it a good run for the money). Recently I did a spreadsheet outlining the savings http://goo.gl/AL5Cj. The spreadsheet was done before the most recent AWS price reductions so the m1.small prices aren't right. For worker servers I recommend using PiCloud. You can install anything you want and they only charge you for the time your jobs use. Granted you have to use Python to interface with jobs but does that really matter in the grand scheme. I personally use Python anyways so its a bonus. Depending on the job it might be more appropriate to use AWS' Elastic MapReduce offering. For database servers I recommend using either RDS or Xeround (http://www.xeround.com) a MySQL provider if you have to use a relational database. If you have the luxury of using a NoSQL option then use Cloudant…

Go take a look at IMB's Cloudant. (https://cloudant.com) They have some examples there demonstrate how fast it is at digesting data from 100s of 1000s of records. It has to do with how the data from searches are stored in a B-tree. I won't pretend to know the science behind that but I can tell you it's impressively fast and easy to achieve. And you can use PouchDB to create the client side interface and plug it into a CouchDB server you set up or IBM's Cloudant, or both for that matter, with just a line or two of code. So you really don't even need to setup a database server to get started on you application code. You can create a Cloudant DB and load it with data and get right to work on your client.

Cloudant seems interesting, never heard of them before. The pricing is intriguing. (https://cloudant.com/pricing/) From what I understand, it's free for most small projects but could get very pricey very fast, but then, handling millions of customers is far from being trivial to handle so I'm sure it's worth the price. Thoughts about the Cloudant pricing versus in-house solution (or vps)?

To be clear, our (Cloudant’s) integrated Search product is only available in our paid product or on hosted accounts at https://cloudant.com Open-Source BigCouch does not have Search built in, but you can still use it in conjunction with Lucene or ElasticSearch (integration is straightforward).

Cloudant has a number of big customers who have been in production for about a year on it. Here is a page where they list a few: https://cloudant.com/#!/company/customers

They also have a pretty impressive team: http://cloudant.com/#about

Wow, a fully Ajax front page (http://cloudant.com). Looks nice but a bit overkill?

Have you looked at Cloudant (https://cloudant.com/)?

This is actually a subject that is surprisingly simple. For package management we have been using Kan.so for npm style package management and deployment (pushing our app to a design doc). I would like to explore just using npm for this, but Kan.so got me there quickly and I learned a lot from the project. Check https://cloudant.com/blog/app-management/ for more. This has worked out really well for us and is easily managed across our team. We all have our own data and just share the code across Github then push with Kanso. Anytime we need example data we just replicate from each other then repush our couchapp and viola. CORS support has not been an issue as we access our app on Cloudant directly through our couchapp then it does the work. On the backend we have nodejs listeners managing tasks through state that Couch cannot handle (emails, signups ...). If you need CORS support on the front end I suggest replicating your data to AWS and having them sync or better yet asking yourself if you really need it in the first place. Feel free to hit me up @hatchware if you have anymore questions and subscribe to [email protected] as it is a very active community.

To be fair CouchDB is very explicit that to get any sort of performance, everything must be a view. "Ad-hoc queries" (i.e. queries that are written on the fly instead of uploaded as a view) are clearly stated as "for development only". Where CouchDB really falls flat is for write-heavy applications. The default configuration in CouchDB is to not reindex a view until it has been read. When a read occurs, any new data in a view that was added since the last read must be re-indexed by executing the map/reduce functions on that data. If you're writing frequently to CouchDB but not reading a lot (as in a data warehouse) the first query you run is going to be extremely slow, since it will need to run map/reduce on a lot of new data. CouchDB doesn't distribute work to multiple nodes like Hadoop, and I've found even simple reduce functions to slow down re-indexing by a factor of 10. I think CouchDB has settings now to update the index on commit, or you could always run a cron job to regularly query the view and force a reindex, but it's still going to be slow. BigCouch (https://cloudant.com/#!/solutions/bigcouch) might be a potential choice for data warehousing, since it advertises full compatibility with the CouchDB API but offers distributed map/reduce like Hadoop/Hive/etc. I haven't used it though.

In fact, there is http://cloudant.com/ from YC.

Cloudant did it for their clustering implementation of CouchDB. See here: https://cloudant.com/blog/run-dmc-explains-network-partitions/

I'd suggest using PouchDB[0] if on a web client, and Cloudant [1] as the backend. Spin up a simple heroku app using Superlogin[2] and you have full Auth, an offline database, sync and now the possibility of conflicts resolution. If developing on mobile clients just use Cloudant Sync for iOS[3] or Android[4]. [0]: http://pouchdb.com/ [1]: https://cloudant.com/ [2]: https://github.com/colinskow/superlogin [3]: https://github.com/cloudant/CDTDatastore [4]: https://github.com/cloudant/sync-android

DB2 also implements the MongoDB query language: http://www.ibm.com/developerworks/data/library/techarticle/dm-1306nosq… So does CouchDB: https://cloudant.com/blog/couchdb-and-mongodb-let-our-query-apis-combi…

Also Cloudant recently came out with their own libraries for iOS and Android (https://cloudant.com/blog/introducing-cloudant-sync), and in principle they should all be compatible with PouchDB/CouchDB. So I like to think we have options.

The video corresponding to those slides is available here: https://cloudant.com/cloudantcon-videos/

For an overview of the query syntax, see https://cloudant.com/blog/introducing-cloudant-query/ Note: I work for Cloudant.

There is another produce coming out from Cloudant very similar, but with even more friendly api IMHO https://cloudant.com/product/cloudant-features/sync/

Here is more info on Cloudant Sync: https://cloudant.com/cloudant-sync-walk-thru

There's also Cloudant Sync based on CouchDB, which is also open-source and for both iOS and Android. https://cloudant.com/blog/introducing-cloudant-sync/ But how do we choose between CouchBase and CouchDB sync?

There is Cloudant, their DB shares the protocol with CouchDB -- a clean restful API. https://cloudant.com/product/cloudant-features/restful-api/

I'm not sure about that. The doc [0] says that there are some static files that should be distributed by whatever HTTP server you want (they suggest python's simple http server) After that, it's all js in the browser. Edit: I see what you mean: Pouchdb-server [1] is a couchdb-compatible server that speaks the same sync protocol. If you don't want to install it, you can work with any hosted couchdb-speaking cloud instance, such as iriscouch [2] or cloudant [3] [0] http://pouchdb.com/getting-started.html [1] https://github.com/nick-thompson/pouchdb-server [2] http://www.iriscouch.com/ [3] https://cloudant.com/

Cloudant will soon offer geo-spatial queries. Its in beta now: https://cloudant.com/product/cloudant-features/geospatial/.

It's because that URL erroneously has the left-to-right control character[1] at the end, which encodes as https://cloudant.com/%E2%80%8E - a missing page on that website. [1] http://en.wikipedia.org/wiki/Left-to-right_mark

Yes, thank you. Just after I posted my question here I realized that much of what I was wondering about was also answered in another post on the Cloudant blog: http://blog.cloudant.com/technical-look-at-cloudant-search/

At least in the case of Cloudant (CouchDB) the servers are in the same facility as AppHarbor. http://blog.cloudant.com/dot-net-couchdb-cloudant-appharbor/

There is a recently open-sourced fork of CouchDB that does the sharding thing. With this addition to the capabilities, CouchDB really can scale up to hundred machine clusters. http://blog.cloudant.com/cloudant-core-is-open-source

as Chris said, it's more like the other systems. See http://blog.cloudant.com/dynamo-and-couchdb-clusters for more information on our clustering.