ClusterHQ Blog

ClusterHQ - https://clusterhq.com/ - San Francisco & Bristol, United Kingdom. Remote working, relocation (VISA) and London positions available. ClusterHQ produces the open-source container data management tool Flocker. We help connect Docker containers with reliable persistent storage, helping to close the gap on getting Docker containers into production. Flocker is a cluster management tool written in Python (Twisted), and we have positions available for all levels of programmers and devops engineers. We believe strongly in TDD, automated testing, quality, collaboration, great colleagues and treating our employees like adults. We have a great work-life balance, flexible hours, fun offices and plenty of opportunity to travel, if you want it! We also have very generous maternity & paternity, on top of the usual full-suite of benefits and great pay packages that you would expect from an established, well-funded startup. Get in touch at https://clusterhq.com/careers/ or come say hello at AWS re:Invent or DockerCon EU.

Do you have any specific reasons not to trust ZoL? ZFS-on-Linux devs say it's ready for production[1]. Lawrence Livermore laboratory stores petabytes of data using ZoL[2]. If we're sharing anecdotes, ZoL has served me fantastically for several years. [1] https://clusterhq.com/2014/09/11/state-zfs-on-linux/ [2] http://computation.llnl.gov/newsroom/livermores-zfs-linux-port-hit-it-…

I'm sad to hear this. I loved reading Richard Yao's blog posts about ZFS on Linux. https://clusterhq.com/2014/09/11/state-zfs-on-linux/

I am running ZFS on Linux and it is working great for me. ZFS on Linux is considered production ready[0]. A lot of the showstoppers, like being able to boot from ZFS have been worked out. [0] - https://clusterhq.com/2014/09/11/state-zfs-on-linux/

In docker-land you've got Flocker. I guess you could version the container with tags... https://clusterhq.com/

Hey, Luke here, CTO at ClusterHQ. As well as being proud of this company milestone, I can't wait to get powerstrip (https://github.com/clusterhq/powerstrip) and flocker (https://github.com/clusterhq/flocker) working together so that we can start composing flocker with other docker ecosystem tools. It will be a great day that you can have a docker swarm with flocker and weave. Or kubernetes with flocker and socketplane. Or even mesosphere with rancher and calico ;) We've got some pretty neat demos of powerstrip prototypes up here, fwiw: http://clusterhq.com/blog/powerstrip-prototype-docker-extensions-today… Any questions just shout! Cheers, Luke

No idea how Docker Swarm handles it, but that the data locality problem for ephemeral containers seems to be the central focus of https://clusterhq.com/ (makers of Flocker).

I am happy to hear that. While I certainly think ZFS is the best filesystem available for storing this kind of data, i would like to add a word of caution that ZFS is not a replacement for backups. I elaborated on this in one of the supplementary blog posts: https://clusterhq.com/blog/file-systems-data-loss-zfs/#disk-failure

Yes. I'm running ZFS on Linux and while ZFS is really great it's not really good integrated in the kernel and sometimes pretty unstable at least in my rather esoteric scenario... Despite other claims FreeBSD suffers similar problems.. (https://clusterhq.com/blog/complexity-freebsd-vfs-using-zfs-example-pa… Other problems are that jails are fine but there is no disk I/O limitation possible... I've also thought about FreeBSD and while pkgng is really great it's a similar problem.. I'm stumbling upon bugs or untested things and I'm unable to contribute time to fixing them.

What do people generally think about Docker Swarm ? The new deployment using .yml files is pretty cool : https://www.infoq.com/news/2017/01/docker-1.13 In fact, IMHO kubernetes has tried to do something similar with .. but it is not engineered ground up for simplicity. Which is why it has MULTIPLE tools for this - minikube, kubeadm, kompose - but nothing matching the ease of use of docker and its yml files. The last survey showed 32% of the polled used Docker Swarm versus Kubernetes' 40% - and this is back when Docker Swarm was highly unstable. https://clusterhq.com/2016/06/16/container-survey/#kubernetes-is-the-m… Are people here using Swarm ? what have your experiences been like.

It would be interesting to hear what are the alternatives now to what they were trying to do with [flocker](https://clusterhq.com/flocker/introduction/). The post seems to make the point other alternatives came up that removed their competitive advantage. Anyone here either using flocker or the other alternatives?

For reference, here is the post from 12/15 where the statement, " I can promise that 2017 will also be a big year for the company" was made by the CTO. Having the CTO make that statement, knowing that the company was shutting down, seems odd. Which implies it was not an orderly shutdown. https://clusterhq.com/2016/12/15/container-predictions/

ZFS is actually one of those features that are officially supported on Ubuntu but not on Debian. As far as I know, Ubuntu 16.04 is the first popular Linux OS to support ZFS out of the box.[1] As the article notes, ZFS on Linux has been production-ready and stable for over three years. For in-depth info see [2]. [1] https://wiki.ubuntu.com/ZFS [2] https://clusterhq.com/2014/09/11/state-zfs-on-linux/

This is also something ClusterHQ are working on with their Flocker product. https://clusterhq.com/flocker/introduction/

Yep, and we think ZFS on Linux is ready real production use https://clusterhq.com/blog/state-zfs-on-linux/

From the horses mouth or so to speak: https://news.ycombinator.com/item?id=9003519 https://clusterhq.com/blog/series-a-funding/

Of course you can't compare the trouble Sun originally had in creating ZFS with the difficulty of porting it to Linux when it has been used for years and you can built on stable code and learn by the other implementations. This comparison is quite unfair and lacking in substance. It's not like the ZoL guys had to start from scratch and reverse engineer It depends of course, I would always choose FreeBSD over Linux when it comes to ZFS, but then again I would always choose a BSD-OS regardless of whether I want to use ZFS or not. Take it certainly with a grain of salt, but the developers themselves announced ZoL to be production ready in 2013/03[1], Richard Yao (also a developer of ZoL) argued in a blog post in 2014/09[2] that ZoL was stable and production ready and elaborated on this. Some interesting comments also are here[3]. And all this also was quite a while ago, ZoL has been used by users for years now, it's come a long way, I have used it to some extent a while ago and had no bad experiences. In the end everyone has to decide for themselves what "production ready" really means, and how high the threshold is to deserve that label. But I think that you can reasonably make the argument, that it is... [1] https://groups.google.com/a/zfsonlinux.org/forum/m/?fromgroups#!topic/… [2] https://clusterhq.com/2014/09/11/state-zfs-on-linux/ [3] http://linux.slashdot.org/story/14/09/11/1421201/the-state-of-zfs-on-l…