Meltwater Engineering
underthehood.meltwater.com · Company blog · on HN since 2015
Meltwater's engineering team, writing about running large Elasticsearch clusters, query load balancing, nginx API testing, and mob programming.
What HN saysHN commenters link its Elasticsearch posts on wildcard handling and query routing, ask engineers about query distribution, and note an ad blocker blocking the site.
25HN people shared it
4linked it in comments
32stories submitted
896story points
7 months agolast shared
2015people per year2026
Most-shared posts
- 1Mob Programming – The Good, the Bad and the Great3 people · 313 points · first shared 2016 · 93 comments on HN
- 2Running a 400+ Node Elasticsearch Cluster3 people · 23 points · first shared 2018 · 1 comments on HN
- 3Using Machine Learning to Load Balance Elasticsearch Queries3 people · 10 points · first shared 2018 · 1 comments on HN
- 4Optimal Shard Placement in a Petabyte Scale Elasticsearch Cluster2 people · 154 points · first shared 2018 · 30 comments on HN
- 5Red Hat memory optimisations and Riak performance2 people · 44 points · first shared 2015 · 6 comments on HN
- 6Simple effective way to test your #nginx API gateway2 people · 6 points · first shared 2017 · discussion on HN
- 7How we optimized wildcards when upgrading our 3PB Elasticsearch cluster2 people · 4 points · first shared 2022 · discussion on HN
- 8Synthetic Monitoring: A Case Study of the Meltwater API2 people · 2 points · first shared 2018 · discussion on HN
- 9We upgraded an old, 3PB large, Elasticsearch cluster without downtime1 people · 149 points · first shared 2022 · 26 comments on HN
- 10Dynamic Route53 records for AWS auto scaling groups with Terraform1 people · 87 points · first shared 2020 · 36 comments on HN
- 11Supercharging the Elasticsearch Percolator1 people · 40 points · first shared 2015 · 4 comments on HN
- 12Locality-Sensitive Hashing in Elixir1 people · 18 points · first shared 2019 · discussion on HN
- 13Getting the most out of your RabbitMQ cluster1 people · 10 points · first shared 2016 · discussion on HN
- 14Risk-Free Deployments with Immutable Web Apps1 people · 8 points · first shared 2018 · discussion on HN
- 15Immutable Web Apps and NPM – A recipe for stable deployments1 people · 7 points · first shared 2021 · discussion on HN
What HN people wrote when they linked it
metasean · 2016-06-08
FWIW, I'm getting:
uBlock Origin has prevented the following page from loading:
http://underthehood.meltwater.com/blog/2016/06/01/mob-programming/
Because of the following filter
||meltwater.com^
karlney · 2023-01-26
Shameless plug, but one of the other blog posts in the above series explain how we addressed high and unpredictable memory load in relation to wildcard searches.
https://underthehood.meltwater.com/blog/2022/11/25/how-we-upgraded-an-…
After those changes were made then we haven't had any major issues with heap spikes. But it depends on a lot of factors, how your data looks, how your queries look and what aggregations that are made so its very hard to give some always applicable advice.
boulos · 2018-11-09
Since I see meltwater folks in this discussion, can you say a bit about the distribution of your queries? In the other blog post about routing [1], you mention that most are a few ms but others are minutes and more.
What’s the rough break down? Are most users performing queries “scoped” to a fairly small subset?
[1] https://underthehood.meltwater.com/blog/2018/09/28/using-machine-learn…
stiangrindvoll · 2018-01-31
Generally when running nginx as an api proxy it can be troublesome to get the routing correctly the first time, therefore testing is required.
This repository showcases how to do nginx api-gateway route tests using RSpec.
This repository also got a blog to explain it in detail:
http://underthehood.meltwater.com/blog/2017/12/12/lightweight-tests-fo…