Google Cloud Platform Blog

I have about 5 years experience with both AWS and Google Cloud (App Engine, Google Compute, ..). I agree that AWS has more services to offer than Google. Initially Google took the route of the App Engine and fell behind AWS. However, recently its Compute Engine and other related services like Container Engine, Datalab (IPython, Jupyter as a Service), Big Query, Dataproc, Pub/Sub and more started outshining their AWS counterparts. Google's network is way faster than AWS. AWS network is too slow and flaky. For anyone taking the route of microservies, Google Cloud offers a huge advantage. Google new cloud console (which is still in beta) is sleek and sexy. Google's cloud services are a pleasure to use compared to AWS (other than a few pain points). I recommend small and medium Cloud-based shops to relook into Google Compute again. Google Cloud is adding services to its arsenal at a super fast rate (much faster than AWS). Just take a look at their engineering blog: http://googlecloudplatform.blogspot.com/. In a year I see google Cloud overtake AWS in terms of the capability.

I'm looking at GAE as a AWS alternative too, so what follows is not complete, just a few observations. Here is the run down on the runtime services available in the Java SDK (I think that it and Python are the most complete, not everything is available for Go last I checked): https://developers.google.com/appengine/docs/java/apis This is a good supplement: https://developers.google.com/appengine/tools_tips So queues and email are in, messaging might be served by Google Cloud Messaging or the recently announced Trello partnership. I don't think anything quite like CloudWatch or Route53 is available. I think of GAE as more like Heroku + add-ons than a straight AWS competitor so I wouldn't expect them to be entirely equivalent. On the other hand more control over the runtime lets them offer things that are entirely out of scope for AWS (channels). As far as updates, the blog shows the the SDK gets new features fairly frequently: http://googlecloudplatform.blogspot.com/

Yes, compare this to a typical Amazon high quality blog post: http://googlecloudplatform.blogspot.com/2016/02/Google-and-Red-Hat-ann… You have no idea what is actually being offered after reading that. Is it a managed service, a recommended configuration of what. Almost every Amazon blog post includes a walk through on how you actually use whatever they are talking about. To be fair, this post does link to more documentation, but it's general purpose and describes a variety of solutions.

I don't agree that AWS and Azure are actually competitively priced with Compute Engine anymore. Following our price cut last May and our custom VM shapes, we're often 40% cheaper: http://googlecloudplatform.blogspot.com/2016/01/Happy-New-Year-from-Go… that's a massive lead (and it's sadly not commonly known). So I agree with you, we need to do better in the perception war, but Compute Engine is hands down the leader in cloud pricing. Disclosure: I work on Compute Engine (and care a lot about our prices).

> 1. Learn the basics of how to architect a small distributed system that is fault tolerant, resilient and reliable. You can't learn it all in 11 days, but man you can make a big dent and it is to me one of the most fun and challenging problems to solve. There was a great tutorial for something like this that google published last month using kubernetes and docker that involves setting up several minecraft servers in a cluster. Would be a great into to docker/kubernetes http://googlecloudplatform.blogspot.com/2015/11/containerizing-in-the-…

Not all ACID transactions are equal. This is just a key-value store-like test. It shows the potential to scale, yet nothing regarding performance in real word. 32 c3.2xlarge instances have 1920GB memory. Given 1 billion 16B+ 8..100B values the whole dataset fits just into memory. The Cassandra test mentioned [1] sustained loss of 1/3 instances. That's very impressive! Would love to see how F-DB handles this type of real-life situation (hint hint for follow up blog post). [1] http://googlecloudplatform.blogspot.cz/2014/03/cassandra-hits-one-mill…

Thanks for sharing your thoughts. Great points and insights. Following up a bit.. Google just announced Kubernetes[1], an open source container manager. Also, Eric Brewer is now on Docker's Governance Committee[2] to help push for open container standards. Seems like a good step forward. [1] https://github.com/GoogleCloudPlatform/kubernetes [2] http://googlecloudplatform.blogspot.com/2014/06/an-update-on-container…

Anything that can process 1MM req/sec gets upvotes from me. 1 million requests per second is 20 times greater than the throughput in last year’s Eurovision Song Contest, which served 125 million users in Europe No, r/s != Gb/s Not sure what the point you think you are making is here, but the Eurovision site was tested to 50,000 req/sec[1]. 50000 * 20 = 1MM [1] http://googlecloudplatform.blogspot.com.au/2013/05/how-scalr-helped-sc…

Part of the problem here could definitely be communication. For whatever reason, it's been incredibly hard to follow what is going on with the Docker project. Keeping info in one place would definitely help. I'm not able to find any details that support what you're saying. It seems that in June Eric Brewer was still publicly asking for libcontainer to merge in LMCTFY support. I definitely could be wrong, but looking at this screenshot from Eric's June talk it looks like they were still trying to get it merged: https://pbs.twimg.com/media/B4DKb51CYAA687s.jpg:large In June, Eric Brewer posted this: We’ve released an open-source tool called cAdvisor that enables fine-grain statistics on resource usage for containers. It tracks both instantaneous and historical stats for a wide variety of resources, handles nested containers, and supports both LMCTFY and Docker’s libcontainer. It’s written in Go with the hope that we can move some of these tools into libcontainer directly if people find them useful (as we have). http://googlecloudplatform.blogspot.com/2014/06/an-update-on-container…

I hear you express this sentiment often. Could you explain your thoughts on this? If anything, I think the stories are the problem. Cool hacks and interesting science often fail to get the 2-3 upvotes they need to escape the new page. On page 2 of "New", I've found the following fairly decent stories that died recently: http://simplystatistics.org/2014/03/20/the-8020-rule-of-statistical-me… http://googlecloudplatform.blogspot.co.uk/2014/03/cassandra-hits-one-m… Yet both of these are vastly more useful than Julie Horvath's "omfg, I had a workplace conflict" or "Yay basic income, no facts, no reasoning, yay" (currently 380 pts, 441 comments, was #1 for a while today). I'd suggest that it's far more important to improve the stories than to improve the comments. The comments on a story like "JDK 8 Release Notes" or "Circuit Breaker" (about a distributed computing design pattern) are actually quite good.

AppEngine is Google Cloud Platform's PaaS offering. Its the oldest product on Google Cloud Platform, but its not one of the ones mentioned that Spotify is using (the Spotify post doesn't talk much about the products, but Google's [0] does.) The Google Cloud Platform services they are identified as using are: Cloud Storage, Compute Engine, Cloud Datastore, Cloud Bigtable, Direct Peering, Cloud VPN, Cloud Router, Cloud Pub/Sub, Cloud Dataflow, BigQuery, and Cloud Dataproc. So, its not surprising that they might be doing something that Google AppEngine doesn't support particularly well. [0] http://googlecloudplatform.blogspot.com/2016/02/Spotify-chooses-Google…

Look no further (for item 1 of 2 anyway) — http://googlecloudplatform.blogspot.co.nz/2015/12/the-next-generation-…

Also on this: http://www.wired.co.uk/news/archive/2016-02/23/spotify-move-to-google-… and http://googlecloudplatform.blogspot.com/2016/02/Spotify-chooses-Google…

Note that this proposal is being back not only by Google, but also Cloudera, Data Artisans, Talend, Cask, PayPal, ... Some other posts on the announcement: http://googlecloudplatform.blogspot.com/2016/01/Dataflow-and-open-sour… http://blog.cloudera.com/blog/2016/01/spark-dataflow-joins-googles-dat… http://data-artisans.com/dataflow-proposed-as-apache-incubator-project… http://blog.cask.co/2016/01/cask-anticipates-googles-dataflow-to-flour…

Google added their 4th region - US East - in October last year[1] but it's true that this is poor effort compared to Azure and AWS. I wrote an article for InfoWorld about this at the end of last year: http://www.infoworld.com/article/3008617/cloud-computing/global-locati… and have a map of locations at https://www.google.com/maps/d/viewer?mid=zV7qz2PQ6Sz4.kJHcuwnR7RZ4 [1] http://googlecloudplatform.blogspot.co.uk/2015/10/Bringing-Google-Clou…

Google's SDN is seriously massive. Petabit of bisectional bandwidth: http://googlecloudplatform.blogspot.com/2015/06/A-Look-Inside-Googles-… One major difference between Google and AWS is that Google will carry your packets between data centers on its backbone by default. Google will also carry packets as close to the customer as possible, whereas AWS will dump it off as quickly as possible. So, even between Google Cloud and AWS it's not an apples to apples comparison.

Indeed it's very unclear, though some little bits from a previous announcement[1]: "bebop is a new development platform that makes it easy to build and maintain enterprise applications." I guess it was vague on purpose[2]: "[bebop] had not publicly said anything about its technology, operating in so-called stealth mode." [1]: http://googlecloudplatform.blogspot.tw/2015/11/investing-in-our-busine… [2]: http://www.nytimes.com/2015/11/20/technology/google-picks-diane-greene…

Announcement: http://googlecloudplatform.blogspot.com/2015/12/Google-Cloud-Vision-AP…

TL/DR - Preemptible VMs are the same as regular instances except for one key difference - they may be shut down at any time [1, 2, 3]. Similar to early AWS spot instances from what I gather (minus the bidding exchange). [1] http://googlecloudplatform.blogspot.ca/2015/05/Introducing-Preemptible… [2] https://cloud.google.com/compute/docs/instances/preemptible [3] https://news.ycombinator.com/item?id=9564122

PHP as well: http://googlecloudplatform.blogspot.jp/2015/06/App-Engine-for-PHP-is-g…

It is linear now - see http://googlecloudplatform.blogspot.com.au/2015/05/Got-a-second-A-leap…

That's how Google does that: http://googlecloudplatform.blogspot.com/2015/05/Got-a-second-A-leap-se… . The drawback is that their clock will be "wrong" for most of that day.

There's a couple of posts on the GCP blog where they compare equivalent as possible setups, comes out a bit cheaper than AWS. http://googlecloudplatform.blogspot.com.au/2015/01/understanding-cloud… http://googlecloudplatform.blogspot.com.au/2015/04/understanding-cloud…

Vendor lock-in isn't especially strong. I wrote the original Twilio/PHP/AppEngine docs almost two years ago: http://googlecloudplatform.blogspot.com/2013/08/getting-started-with-t…

Blog post doesn't have much information not found in http://googlecloudplatform.blogspot.com/2015/06/App-Engine-for-PHP-is-…

So their benchmark of Cassandra against BigTable doesn't even match their previous benchmark of Cassandra. http://googlecloudplatform.blogspot.com/2014/03/cassandra-hits-one-mil… How did the latency for Cassandra on their cloud platform increase by 200ms from a year ago?

This has been around for quite sometime[1]. It's also cool to note that there are usually free credits involved in most Google Cloud competitions and some Google Udacity courses. Off the top of my head, I think I might have enjoyed a total of about $3000 since 2013. Oh and there's the $100000 for Startups[2] that'd be cool to get at some point in time. 1: [http://googlecloudplatform.blogspot.com/2014/11/google-cloud-platform-… 2: [https://cloud.google.com/developers/startups/]

1. Google Container Engine runs Docker containers, the same as other providers like AWS EC2 Container Service and Joyent Compute Service. 2. I don't understand what you mean. 3. You run your containers on virtual machines dedicated to you, so you rely on the hypervisor for security. 4. Cgroups is one of the technologies that linux containers are built on. What is your point? 5. Google is building their container service on docker and kubernetes [0]. Both of these are open source, already adapted by multiple providers, and possible for you to run yourself. [0] http://googlecloudplatform.blogspot.com/2015/01/everything-you-wanted-…

Maybe not exactly the same? The Docker technology was a by-product of the dotCloud product, which spun-off to create the company Docker, Inc to focus on Docker. Docker Inc sold dotCloud to the Berlin-based company cloudControl. This appears to be a facelift coming from the new leadership. Also, www.dotcloud.com references next.dotcloud.com with this language: "Take the new and improved dotCloud PaaS on Google Compute Engine for a test-drive." Their old FAQ said that containers were pushed to AWS East, so it looks looks like this is one thing that has changed. Actually, yes, here are some details on the move from AWS to Google Compute Engine: http://googlecloudplatform.blogspot.com/2014/12/dotcloud-provides-fast…

Well, Amazon also has a 6 year head start. Compute Engine has only been generally available since December 2013: http://googlecloudplatform.blogspot.co.uk/2013/12/google-compute-engin… App Engine was released a long time ago, around the same time as AWS, but they aren't comparable products. It seems clear that the market chose the AWS technology since it was more compatible with existing code.