The HFT Guy
thehftguy.com · Personal blog · on HN since 2016
An anonymous engineer from the high-frequency trading world writes about Docker, cloud pricing, databases, and career advice.
What HN saysHN links it for critical posts on Docker in production, AWS versus Google Cloud costs, and NoSQL databases, with commenters citing it in container and database debates.
75HN people shared it
34linked it in comments
90stories submitted
4.2Kstory points
9 months agolast shared
2016people per year2026
Most-shared posts
- 1Docker in Production: A History of Failure (2016)16 people · 142 points · first shared 2016 · 99 comments on HN
- 2How Many Visitors Do You Get from the Hacker News Front Page?8 people · 6 points · first shared 2017 · 3 comments on HN
- 3Google Cloud is 50% cheaper than AWS – The HFT Guy7 people · 11 points · first shared 2016 · discussion on HN
- 4Docker in Production: An Update6 people · 171 points · first shared 2017 · 92 comments on HN
- 5Career Advice and Salary Negotiations: Move Early and Move Often6 people · 55 points · first shared 2017 · 16 comments on HN
- 6Why products are shutdown? The case of Verizon and Yahoo Groups4 people · 15 points · first shared 2019 · 1 comments on HN
- 7GCE vs. AWS in 2016: Why You Shouldn't Use Amazon3 people · 149 points · first shared 2016 · 169 comments on HN
- 8French judge rules GPL license must be enforced via contract law, not copyright3 people · 75 points · first shared 2020 · 50 comments on HN
- 9A Haven for Fraud and Stolen Cards or How My Wife Was Forced to Quit Revolut3 people · 9 points · first shared 2020 · 5 comments on HN
- 10The Differences Between Splunk, Kibana and Graylog. – The HFT Guy3 people · 9 points · first shared 2020 · discussion on HN
- 11The 187M Dollars Gmail Bug3 people · 4 points · first shared 2017 · 1 comments on HN
- 12Scrum vs. Kanban: Aren't they the same thing?3 people · 1 points · first shared 2017 · discussion on HN
- 13The most remarkable legacy system I have seen2 people · 526 points · first shared 2020 · 182 comments on HN
- 14GDPR Violation: Scribd acquires PII on 500M users in a deal with LinkedIn2 people · 333 points · first shared 2020 · 124 comments on HN
- 15French Appeal Court affirms decision that copyright claims on GPL are invalid2 people · 297 points · first shared 2021 · 198 comments on HN
What HN people wrote when they linked it
unixhero · 2022-02-18
LXC has been so stable and great to work with for many years. I have had services in production on LXC containers and it has been a joy. I can not say the same about things I have tried to maintain in production with Docker, in which I had similar experiences to [0], albeit around that time and therefore arguably not recently.
For a fantastic way to work with LXC containers I recommend the free and open Debian based hypervisor distribution Proxmox [1].
[0], https://thehftguy.com/2016/11/01/docker-in-production-an-history-of-fa…
[1], https://www.proxmox.com/en/proxmox-ve
throwaway3157 · 2020-02-18
According to a recent post [1], "Pickle, number #1 source of critical vulnerabilities in python applications since 1991" which was interesting & newsworthy to me. I've used Python for a long time and was unaware of this
[1] https://thehftguy.com/2020/02/18/jwt-is-awesome-heres-why/
guitarbill · 2018-02-18
IBM acquired Softlayer, because otherwise their "cloud" offerings were basically marketing wank. Softlayer offered dedicated hardware, which a few years ago might have been appealing because of compliancy issues or lack of beefy machines on other cloud providers. In classical fashion, IBM snoozed and the competition has caught up (<insert IBM being morons rant here>). Great post if you want more detail: https://thehftguy.com/2018/01/15/the-inevitable-demise-of-ibm-softlaye…
user5994461 · 2017-06-24
It's not trolling. MongoDB really doesn't have a purpose. There is nothing it is good at.
https://thehftguy.com/2017/03/29/whats-the-best-nosql-database-cassand…
logging => elasticsearch
distributed cache => memcache
analytics => any SQL database for low volume (< 1TB). any data warehouse database for high volume (> 10TB).
nodesocket · 2017-03-15
> Also, they don't have a sales force like Amazon. A bunch of non-engineer humans who do nothing but convince other people to buy their product.
Here's the catch, the "bunch of non-engineer humans" as you called them are not stupid though. All around these people are watching the landscape and market. They are aware that GCP has a superior pricing structure and can be up to 50% cheaper[1]. Therefore my theory is that GCP can continue to offer lower prices, greater governance, superior performance, and solid support, and their (sales+engineering) force can be in-bound driven instead of out-bound like AWS.
[1] - https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-than-aws/
johansch · 2017-02-28
Google Cloud if you're looking for something similar. It's just so much better and cheaper. I think a lot of the resistance here towards that kind of move is just because people are inherently lazy and they aren't paying the bill themselves.
(I'm guessing a relatively large part is also selfish attachment to the market leader because of employment reasons. I hate wasting money, both for myself and for my employer, so I don't really understand this kind of thinking - but I do understand how it could flourish in a venture capital-rich time/locale.)
I also recommend reading:
https://thehftguy.com/2016/06/15/gce-vs-aws-in-2016-why-you-should-nev…
ferdek · 2020-10-23
I believe the way you put it describes this part in the DMCA takedown notice the best way possible. Bringing up European laws on intellectual property and mixing it with US "copyright" is comical to me, especially after reading this [0] article on French law recently:
> France doesn’t have copyright. Sorry for US readers, your “copyright” is nonsense that doesn’t apply here, though the word “copyright” can be seen misused verbatim once in a while.
I believe this interpretation of "copyright" is all similar around Europe. US copyright law is all about "right to make copies", be it software or music/video and it just doesn't make any sense on the other side of the globe.
In the country where I live I can purchase a music or video file and legally make unlimited copies for my personal use. It is also legal to download it from any other source as long as I can prove that I paid for it some time ago in the past. Torrents are illegal as the protocol forces me to share the file, and sharing part is illegal. Downloading is fine xD.
[0] https://thehftguy.com/2020/09/15/french-judge-rules-gpl-license-to-be-…
stephenr · 2017-09-02
See, a lot of people say things similar to this.
But let's think logically: with Go, you have a single binary file, that will run on basically any distribution of Linux, with no external dependencies.
With Docker, you need a lot more than that, and in the case of a Go binary, you have no benefit.
I'd suggest reading through https://thehftguy.com/2017/02/23/docker-in-production-an-update/ for an idea of "Docker in production". Sure, we aren't all running HFT systems, but the issues he documents aren't really specific to HFT - they're more related to having a piece of software you can rely on to work.
Can containers in the generic sense be a useful tool for certain tasks? Sure.
Is Docker the "omg lets put bread around this meat and call it a sandwich" epic moment? No.
The rise in mindshare of Docker is IMO not coincidentally linked to the rise of the bad kind of DevOps: where management fires ops, and gets developers to run their infrastructure.
"I don't need to understand how <insert common Linux infrastructure software> works, I can just run 2 docker commands and it will download me a working image from the internet. What do you mean who created the image and can I trust it? This is the Internet, of course anything I download is trustworthy."
timkpaine · 2024-08-30
No, but you can find some snippets:
https://youtu.be/M9o9SF5-Pzw?si=Nw-UcTN9HJnIo_6d
https://thehftguy.com/2020/07/09/the-most-remarkable-legacy-system-i-h…
newusertoday · 2024-02-22
Here is one more https://thehftguy.com/2017/09/26/hitting-hacker-news-front-page-how-mu…
it would be interesting to compare them
Thaxll · 2023-12-01
I would double check what OP found because his blog is full of inaccuracies, ex:https://thehftguy.com/2023/11/14/the-linux-kernel-has-been-accidentall…
tester756 · 2023-12-01
On the other hand he had very influential and interesting posts like this
https://thehftguy.com/2016/11/01/docker-in-production-an-history-of-fa…
https://thehftguy.com/2017/02/23/docker-in-production-an-update/
pelasaco · 2023-05-16
From the same author, this article looks more interesting https://thehftguy.com/2021/08/30/french-appeal-court-affirms-decision-…
sva_ · 2022-06-23
I think the case you're thinking of was in France:
https://thehftguy.com/2020/09/15/french-judge-rules-gpl-license-to-be-…
naavis · 2021-03-21
Getting to the front page of HN brings a lot more traffic than "a few dozen visitors at once".
https://nicklafferty.com/blog/what-happens-when-you-re-on-the-front-pa…
https://thehftguy.com/2017/09/26/hitting-hacker-news-front-page-how-mu…
throwdbaaway · 2020-09-13
> performing sweeping changes across an entire codebase in lockstep
This thinking right here, sounds like a reason why Google retires a lot more services at much higher frequency compared to other companies?
In https://thehftguy.com/2019/12/10/why-products-are-shutdown-the-case-of… the HFT guy brought up the "5 years upgrade pain" as the reason why a service would be retired, at the time when the pain from upkeeping outweights the gain from revenue.
By regularly making backward incompatible API change, as afforded by the monorepo, while also having engineers moving freely between projects, the upgrade pain cycle becomes way shorter for Google services.
By the way, this line of thought also leaks into public facing source code, with guava being the poster child of breaking backward compatibility, althought it has learned its lesson starting with version 21.
AlexandrB · 2020-09-08
Revolut was featured on HN just this morning: https://thehftguy.com/2020/08/12/a-haven-for-fraud-and-stolen-cards-or…
dimitar · 2020-08-16
Does Solr have what the HFT guy calls 'catastrophic typing' in ES?
https://thehftguy.com/2020/08/04/the-differences-between-splunk-kibana…
avip · 2019-08-17
This guy knows https://thehftguy.com/2017/09/26/hitting-hacker-news-front-page-how-mu…
(he got 20K/day, which for all intent equals zero in my experience)
StreamBright · 2019-03-10
> What’s wrong with Docker? Why would I want to switch to something else?
Security, performance, reliability and following best engineering practices. Everything else is fine.
https://arxiv.org/pdf/1804.05039.pdf
https://hackernoon.com/another-reason-why-your-docker-containers-may-b…
https://www.scylladb.com/2018/08/09/cost-containerization-scylla/
https://thehftguy.com/2017/02/23/docker-in-production-an-update/
http://catern.com/posts/docker.html
nceheil · 2018-10-10
I really appreciated this article:
https://thehftguy.com/2017/01/23/career-advice-and-salary-negotiations…
Hope it helps you the way it has helped me.
known · 2018-08-03
zkomp · 2018-05-02
Well the Title is "Containers".
Punchline is about writing software or glue together things you dont understand.
The mention of that hyped and also hilariously buggy product seems relevant. See https://thehftguy.com/2016/11/01/docker-in-production-an-history-of-fa… for a few years old example of this "gluing together things without understanding"
foo232323 · 2018-02-13
Nope actually just being on there (with a current profile) helped a lot. Currently at least here in Germany there are a _lot_ of open positions in my field. At the moment I (as well as other colleagues I talk to) get multiple requests a week. There are a lot of uninteresting ones, but still..
Also: What's mentioned here: https://thehftguy.com/2017/01/23/career-advice-and-salary-negotiations…
brianwawok · 2018-01-28
I think I would rather use GCE
https://thehftguy.com/2016/06/15/gce-vs-aws-in-2016-why-you-should-nev…
rb808 · 2017-10-30
The HFT Guy has a good blog and did some posts on this. Basically Google Cloud is much cheaper than AWS so you should look at that. There are also other alternatives including IBM SoftLayer which I've never looked at before.
"Run an entire tech company in the cloud, or run only a single [big] project requiring more than 10 servers? Google Compute Engine
Run less than 10 servers, for as little cost as possible? Digital Ocean
Run only beefy servers ( > 100GB RAM) or have special hardware requirements? IBM SoftLayer or OVH"
https://thehftguy.com/2016/06/08/choosing-a-cloud-provider-amazon-aws-…
https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-than-aws/
eicnix · 2017-10-30
Somebody wrote a blog article about this: https://thehftguy.com/2017/09/26/hitting-hacker-news-front-page-how-mu…
obulpathi · 2017-08-21
I don't think so. GCP's bill is usually about 50% of AWS's bill for same application, if you run it full hour (from my personal experiences and from several others as well: https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-than-aws/)… GCP has lot more cost saving features like seamless scalability, custom shapes, sustained discounts and so on. If you workloads span less than hour, GCP can offer more then 50% savings.
alpb · 2017-04-28
Try Google Cloud, it may get even cheaper. https://thehftguy.com/2016/11/18/google-cloud-is-50-cheaper-than-aws/ (disclaimer: I work at Google)
lloydde · 2017-04-08
"Basically, you need to perform a binary search. When you get an offer, you know that you can get at least that. Then you repeat the process a bit higher." https://thehftguy.com/2017/01/23/career-advice-and-salary-negotiations…
Except that isn't a binary search.