Kubernetes Blog

> Docker Compose yml form 3.1 is brilliant What's nice about ECS is that it lets you deploy using Docker Compose files (though you often make minor modifications for ECS) ecs-cli compose --file ecs-compose.yml create However, there is the kompose project to bring Docker Compose to Kubernetes (it's still in Kubernetes Incubator at this point): http://blog.kubernetes.io/2016/11/kompose-tool-go-from-docker-compose-… https://github.com/kubernetes-incubator/kompose

There are a bunch of scale limits in Kubernetes itself that are being chipped away at; the bottleneck is not the scheduler, so plugging a new scheduler in won't make the system more performant. Here's a blog discussing the current state of K8s scale: http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-… Note that Bob Wise at Samsung has been driving some horizontal scale testing, and has got a 1000-node cluster up and running, so that's a current "best case" scale number.

Yes, the community is working hard to make k8s scale to 1000s of machines and beyond. There is a great blog post about the engineering effort from the various communities (k8s, etcd) to make this happen: http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-… In fact someone recently made a video showing a 1000 machine cluster; let me find that. UPDATE: here is the video https://www.youtube.com/watch?v=sgWv9sVTIYQ

I don't think there's a direct quote of k8s devs saying "this is designed for stateless applications" (although there very well may be), but it's pretty evident if you're following. Blog post on the official site, about 9 months old, whose title expresses surprise that k8s may be reasonable for stateful applications. The title is "Stateful Applications in Containers!? Kubernetes 1.3 Says “Yes!”" [0] k8s's "StatefulSets", which emerged in 1.5 (4 months ago, Dec 2016) and were transitioned from the previous feature "PetSets", are still marked a "beta" feature in the newest release (1.6, made a couple of weeks ago). [1] In Nov. 2016, CoreOS issued a blog post with the subheading "Stateless is Easy, Stateful is Hard" while introducing a new pattern called "operators". Operators are intended to help Kubernetes better handle stateful applications (since the habit of k8s admins is to terminate pods casually). These are complicated to implement and haven't been widely adopted; basically, you register a third-party resource type, tell your operator to listen in for pod deletions, and hijacks these commands to ensure that everything is shut down in the correct order and with ample opportunity to save. [2] ----- This is all just the k8s level -- let's not forget that the container engine underlying k8s will delete all changes made to the image by default, and has similar weird issues with stateful applications. Accidentally stop the wrong container on the docker cli? You have…

It's great that hftguy thinks Google Container Engine is stable (I work on it) but I'm sorry to say it's very easy to prove that is, in fact, running Docker on the nodes. You can just SSH into one and see for yourself. Kubernetes was built from the ground up to orchestrate Docker. CoreOS did a lot of work to make it possible to trade rkt in for Docker's engine, and the cri (Container Runtime Interface) is now generalising that so that there is a clear abstraction between the kubelet and the engines it orchestrates. Read about it here: http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-k… If you want to do things that are different to what we provide support for on GKE as a Managed Service (tm), you're able to run your own Kubernetes clusters on GCE. (We do let you run a Kubernetes alpha version, but only on non-supported clusters that self-destruct after 30 days.

Kubernetes absolutely does not force you to use the docker runtime. In fact, there has been a lot of work to avoid this by creating the CRI[0]. Kubernetes also supports extensions like the Third Party Resource or their successor, Custom Resource Definitions. KubeVirt[1] is an example of extending resources to include VMs [0] http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-k… [1] https://github.com/kubevirt/kubevirt

hey! we at bitmovin have been using k8s for quite a while for our infrastructure and on premise deployments. In case you're interested in how we do multi stage canary deployments, check out: http://blog.kubernetes.io/2017/04/multi-stage-canary-deployments-with-…

There is support for automatic certificate rotation in the recently released 1.7. Pretty sure this was also in 1.6, albeit as an experimental alpha feature: http://blog.kubernetes.io/2017/06/kubernetes-1.7-security-hardening-st…

You're probably interested in the Container Runtime Interface [0] 0 - http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-k…

The presentation of patterns here is quite helpful. Is anyone aware of other resources for container design patterns? The k8s blog has some as well: http://blog.kubernetes.io/2016/06/container-design-patterns.html

>Docker should have just licensed and supported the Docker engine to the mass market (their one moat) I give them a lot of credit for noticing that nobody else was capitalizing on the existing container tech. But it did exist already. They didn't invent something new. They just put a nice cli, daemon, and image repository around it. I think they know the "license the engine" bit is a non-starter, as the higher level container orchestration platforms are already abstracting away from it.[1] [1]http://blog.kubernetes.io/2016/12/container-runtime-interface-cri-in-k…

> They haven't really shown an interest in Kubernetes thus far Except for helping with Windows Server support for kubernetes[0] and supporting kubernetes in Azure Container Services[1] [0]: http://blog.kubernetes.io/2016/12/windows-server-support-kubernetes.ht… [1]: https://azure.microsoft.com/en-us/blog/kubernetes-now-generally-availa…

Yes, see the blog http://blog.kubernetes.io/2016/07/cross-cluster-services.html

Two of the three Kubernetes co-founders (Joe Beda and Craig Mcluckie) co-founded Heptio. The third (Brendan Burns) left Google for Microsoft, where Kubernetes on Azure reached GA today [0]. [0] http://blog.kubernetes.io/2017/02/caas-the-foundation-for-next-gen-paa…

Hi, on the kubernetes blog[0], there's a command: `fission function create --name hello --env python --code hello.py --route /hello` The `function` function, doesn't seem support route, and should be broken up into two parts: `fission function create --name hello --env python --code hello.py fission route add --function hello --url /hello` Taking a look at the help menu for function it's not there, I'm running: `fission version 0.0.0` [0]http://blog.kubernetes.io/2017/01/fission-serverless-functions-as-serv…

Kubernetes Dashboard - http://blog.kubernetes.io/2016/07/dashboard-web-interface-for-kubernet…

Fixed URL: http://blog.kubernetes.io/2016/10/Production-Kubernetes-Dashboard-UI-1…

As the proverb goes, "Choose, or someone will choose for you." rkt is getting into Kubernetes as an alternative runtime engine: http://thenewstack.io/kubernetes-chief-we-back-docker-although-appc-mi… (2015) http://blog.kubernetes.io/2016/07/rktnetes-brings-rkt-container-engine… (2016) Back then it was, "It's sad to see so much drama; Docker engine has much more traction." Remember all of that? The story is now, "hey, rkt lets us test out our framework for supporting alternative runtime engine". There's a shifting attitude there. Goodwill from the community is eroding. I don't know how long this window will remain open, but it won't remain open indefinitely.

see also hypernetes: http://blog.kubernetes.io/2016/05/hypernetes-security-and-multi-tenanc…

Our blog post is live! http://blog.kubernetes.io/2016/07/kubernetes-1.3-bridging-cloud-native… Disclosure: I work at Google on Kubernetes

Kubernetes team uses Vegeta in their 10,000,000 QPS load test. http://blog.kubernetes.io/2016/03/1000-nodes-and-beyond-updates-to-Kub…

ShareThis | Palo Alto, CA | Full-Time | ONSITE ShareThis helps publishers provide a more authentic digital experience. We create widgets that sit on millions of domains and process TB of data for various use cases daily. http://www.sharethis.com/hiring.html - Cloud Platform Engineers to work with Docker and Kubernetes - http://blog.kubernetes.io/2016/02/sharethis-kubernetes-in-production.h… - Awesome Silicon Valley Company Culture

The networking features introduced in Docker are one "standard"; CNI is another "standard" [1]. Weave Net as a product ships with support for the former; the blog post describes work towards support for the latter. No they do not interoperate; there is a detailed argument at http://blog.kubernetes.io/2016/01/why-Kubernetes-doesnt-use-libnetwork… [1] Scare-quotes because all of these things are only a few months old.

They're adding integration in with Docker too for Criu: http://criu.org/Docker Here's a fun example of it in action: http://blog.kubernetes.io/2015/07/how-did-quake-demo-from-dockercon-wo…

"Adding k8s on mesos you get the next-generation cloud native scheduler and the ability to run existing workloads. by running k8s by itself you get a lightweight cluster environment for running next gen cloud native apps." @cmcluck More references: [1] https://mesosphere.com/blog/2015/04/22/making-kubernetes-a-first-class… [2] http://blog.kubernetes.io/2015/04/kubernetes-and-mesosphere-dcos.html [3] http://thenewstack.io/mesosphere-now-includes-kubernetes-for-managing-…

Correct, which is how it slants to the "Borg cube" pun. Cybernetics was the term chosen by Norbert Wiener in the book "Cybernetics," and he traced the word's origin to the greek "kubernetes;" it related to his first example of a cybernetic system, the self-correcting steam-controlled rudder on a ship [http://en.wikipedia.org/wiki/Steering_engine]. (Why the pun? Kubernetes was heavily inspired / guided by Google's internal scheduling tool, which was named Borg (http://blog.kubernetes.io/2015/04/borg-predecessor-to-kubernetes.html)…

See also Google's blog post summarizing Borg -> Kubernetes improvements. http://blog.kubernetes.io/2015/04/borg-predecessor-to-kubernetes.html

DC/OS and k8s have some overlaps, but I wouldn't characterize the two as direct competitors. Majority of DC/OS users are running distributed data services like Spark, Kafka, Elastic, Cassandra, etc. And they're doing this for at least three reasons 1) Pooling these distributed systems instead of running them in separate clusters, 2) The automation for scaling and high availability that Mesos frameworks enable, and 3) Being able to run these data services in any cloud they choose - Google, Azure, AWS, or their own. The 2016 Mesos Survey (n=500) has more details. If you are squarely focused on container orchestration, 62% of DC/OS and Mesos community runs containers in production. Average size of their environments is 200+ nodes. A similar survey of the K8s community showed rough average K8s environment is around 20 nodes. Disclosure: I work at Mesosphere https://mesosphere.com/blog/2016/11/02/apache-mesos-survey-2016/ http://blog.kubernetes.io/2017/01/kubernetes-ux-survey-infographic.htm…

I've been a big fan of ipvlan since it was added to the kernel[1] mainly for the reasons you've outlined and the overall reduction in complexity compared to bridges and overlay networks. It should also be noted that ipvlan offers a bit of an improvement over macvlan because ipvlan also works in places where L2 is not an option (i.e cloud providers). In the post you mentioned one of the drawbacks to adopting ipvlan was the lack of tooling to manage it. To address this issue, and a few others[2], Kubernetes has recently adopted the Container Network Interface(CNI)[3] standard as the solution for managing network plugins in Kubernetes. CNI ships with a few plugins including ipvlan and, maybe more importantly, the ability to allocate and manage IP addresses[4]. Over time CNI should give Kubernetes the flexibility to work well with multiple container runtimes including Docker and rkt, while allowing third party solutions to fill in the gaps around networking. [1] https://www.kernel.org/doc/Documentation/networking/ipvlan.txt [2] http://blog.kubernetes.io/2016/01/why-Kubernetes-doesnt-use-libnetwork… [3] https://github.com/appc/cni/blob/master/SPEC.md [4] https://github.com/appc/cni/blob/master/SPEC.md#ip-allocation