MinIO Blog

There was a 60 days notice? https://blog.min.io/ The only notice I have received is from the blowback of their behavior. Walking away from supporting the open source product at any other time than in the wake of a security vulnerability disclosure would have been received better. Sure they're not legally culpable for fixing anything but they're now obviously untrustworthy as a business partner going forwards.

Here's [1] Graviton shooting out with AMD & Intel, underperforming in some benchmarks, outperforming in others. Here's another [2] showing how the power estimate for graviton is less than half of Intel & AMD which means that it's about 2x better on a power/watt level which is how you measure performance typically. Here's [3] [4] Amazon claiming how their in-house Nitro SSDs keep improving. Here's [5] a comparison of AWS vs other cloud providers, noting how AWS leads in 7/10 metric categories one of which is disk performance and [6] is a similar report showing how AWS nitro disks lead in latency by a factor of 2x. The 2022 report [7] shows more mixed results so things may have equalized, but only because other providers are following Amazon and catching up with their own equivalent solutions, not because they're relying on third-party SSD vendors to build better NVMe drives. But honestly, for things that are factual and as easily googlable as this, I would suggest you do your own research to become informed rather than expect to be hand-fed. [1] https://www.percona.com/blog/comparing-graviton-performance-to-arm-and… [2] https://blog.min.io/intel_vs_gravitron/ [3] https://aws.amazon.com/blogs/aws/aws-nitro-ssd-high-performance-storag… [4] https://aws.amazon.com/blogs/aws/introducing-storage-optimized-amazon-… [5]…

Its worse than I thought: https://blog.min.io/weka-violates-minios-open-source-licenses/ They think they can revoke someone's AGPL license. That's not at all how that license works!

What is the use case for implementing a POSIX filesystem on top of an object store? I remember reading this article a few years ago, which happens to be by the minio folks: https://blog.min.io/filesystem-on-object-store-is-a-bad-idea/

The blog post is quite nice: https://blog.min.io/minlz-compression-algorithm/ But comparisons with zstd & lz4 would be nice. Is someone planning on a wuffs/rust implementation with C API so that it wouldn't be Go only?

An open-source implementation of Amazon S3 - MinIO has had it for almost two years (relevant post: https://blog.min.io/leading-the-way-minios-conditional-write-feature-f… Strangely, Amazon is catching up just now.

Minio used to support this, but removed the feature a while back. It was called “gateway mode”. Sadly, I know that doesn’t help much now… https://blog.min.io/deprecation-of-the-minio-gateway/amp/

Minio used to have a filesystem backend that would store each file as a file on the local filesystem. They removed this, so lots of people still use the last version before that. https://blog.min.io/deprecation-of-the-minio-gateway/

Perhaps even stranger, MinIO have publicly stated they have revoked an Apache 2 license grant to a third party, Weka: https://blog.min.io/weka-violates-minios-open-source-licenses/ Not sure what their counsel is thinking there..

There have been other instances in which exit fees, which is what this amounts to, were considered anticompetitive, e.g. [1] (although this was settled so there is no ruling). Google itself has started to waive egress costs for GCP customers leaving the platform last month, which, according to some sources, is simply a direct consequence of new EU legislation (Data Act) [2], but according to others is in anticipation of wider-reaching EU antitrust investigations [3]. [1]: https://www.agriculturedive.com/news/koch-foods-antitrust-chicken-poul… [2]: https://www.computerweekly.com/news/366566360/Googles-data-egress-offe… [3]: https://blog.min.io/googles-new-egress-policy/

This has not been true since 2021. https://blog.min.io/minio-optimizes-small-objects/

It does appear that their minio instance is the apache version. From the minio allegations the ui in the screenshots matches the pre-AGPL instance that I've kept around which was really just a simple bucket/files manager. I think all post-AGPL versions should be using the new ui announced here[1] in April 2021. The AGPL change was announced 12 May 2021[2]. The newer date in the out of date message could be due to them re-compiling the Apache version themselves. However looking at the warp version in the screenshot, version 3.40 is licensed under AGPL. [1] https://blog.min.io/new-minio-console/ [2] https://blog.min.io/from-open-source-to-free-and-open-source-minio-is-…

>JuiceFS has introduced S3 gateway since v0.11. The feature is implemented based on the MinIO S3 Gateway. The MinIO S3 Gateway was deprecated: https://blog.min.io/deprecation-of-the-minio-gateway/ I don't know if JuiceFS is treating it like a fork that they are maintaining, or if they have other plans.

It doesn't really have to impact performance. The index is generated easily as a side-effect of compression. And the index is only needed if you need to seek. I implemented this as part of the MinIO server. See "Seeking Compressed Files" here: https://blog.min.io/transparent-data-compression/ We choose a compressor without literal compression for a faster baseline, but the concept remains the same.

Original link 404s. It's missing a character at the end. Try: https://blog.min.io/hybrid-cloud-red-hat-openshift/

When Companies Should Go to the Public Cloud + When they Should Return https://blog.min.io/repatriation_wave/

The MinIO code is highly optimized for the md5 hashes using AVX512 https://blog.min.io/accelerating-aggregate-md5-hashing-up-to-800-with-…

As per last week's announcement from AWS about the availability of AMD EPYC cpus, we have repeated the “single socket” test for the EPYC cpu as well. The updated chart is included in the post or you can find it here: https://blog.min.io/content/images/2020/06/single-socket-performance--… It clearly performs and scales a lot better than the Intel CPUs (no doubt also benefiting from the increased memory bandwidth) and at high core counts is very close to the Graviton2. Furthermore, I agree that the “investment costs” for Amazon are almost minute, they might have already earned it back.

I was looking at the single-socket results for what I said above specifically to rule out any NUMA differences: https://blog.min.io/content/images/2020/06/single-socket-performance.p… That really looks like it's just sorting by memory bandwidth. Particularly with Cascade Lake's results vs. Skylake at the ~10 core number, since there's not really any other major difference between those two. Being that memory bandwidth constrained, and then not being NUMA aware, is also quite likely why the dual-socket results are a disaster.

>Isn't RAID parity slightly more space efficient than versioned backups? It depends on your storage array. The more drives, the more space efficient RAID becomes, but RAID is still only a single copy of your data. >Or is there a better way to do redundancy that doesn't involve just replicating entire files to multiple disks Most of the industry is using erasure coding these days (https://blog.min.io/erasure-coding/) which allows for spreading your parity and data across multiple sites. Erasure coding usually runs a layer above the filesystem, as opposed to RAID which typically runs below the filesystem (Snapraid, Mergerfs and others excluded). My personal "backup vault" is a Raspberry Pi 4 with a single 4TB external drive attached. The RPi runs Minio, and all backups are done through the S3 interface or SFTP/SMB. It is not the fastest box in the world, but it backs up (incremental) ~2TB in 30 minutes, which is "fast enough". It consumes on average 4W, which means even with worst case electricity prices of €1/kWh (which we saw last winter), it costs less than €3/month. For comparison, my NAS consumed around 50W, and at €1/kWh, that would cost €37/month in electricity alone, and then you need to add the cost of the actual hardware itself. I switched off the NAS, and purchased ~10TB of cloud storage (main storage and backup storage at two different locations) for €20/month, and keep sensitive stuff encrypted with Cryptomator.

It is astonishing as to how marketers from the three big clouds have successfully convinced people that there is no middle ground between operating your data center and using one of their cloud services. Here's what you do: 1. Buy two servers equipped with a large amount of storage. There are many providers[1][2] that offer this. (You could technically do it with one, but two allows you to have high availability and prevent data loss in case of accidents, imagine `rm -rf *`.) 2. Use Minio to set up erasure-coded storage on one of the servers, and replicate objects[3] to the other server. 4. Use Cloudflare (because of their open source sponsorship program[4]) to serve the objects from the primary server, and when performing maintenance tasks, fall back to the secondary. You could also serve objects directly out of the primary/secondary, but using Cloudflare takes care of low-effort DDoS attacks by hiding the origin, and provides some caching which helps prevent hitting the egress limits of the server provider. [1] https://www.netcup.eu/vserver/vstorage.php [2] https://www.hetzner.com/dedicated-rootserver [3] https://blog.min.io/active-active-replication/ [4] https://blog.cloudflare.com/cloudflare-new-oss-sponsorships-program/

Despite the HN bubble, there are a lot bigger markets for pro users than developers. I would venture to say that from a profit motive, it makes little sense to bring development to the iPad. There is no money in development tools. For places that must have bespoke internal apps, a lot of them are moving toward dumb clients and some type of Citrix solution. In terms of “sheer numbers”, the desktop market is minuscule, the laptop market could easily go to ARM if MS could ever figure out a good transition strategy, and the server market runs Linux where power is big as important as overall power consumption. Don’t you think that Intel would rather be in ARMs position than what it is now? I don’t have numbers on the server market, but outside of that Apple ships more chips than Intel. Edit: this was just submitted to HN showing the performance of Intel chips to Amazon’s custom designed ARM chips. https://blog.min.io/intel_vs_gravitron/