Ilya Grigorik
igvita.com · Personal blog · on HN since 2007
Ilya Grigorik, a Google web performance engineer, writes on networking, latency, storage engines, Ruby internals, and blockchain basics.
What HN saysHN commenters call it one of their favorite blogs, praise his networking expertise, and rate his Minimum Viable Block Chain the best blockchain introduction.
177HN people shared it
99linked it in comments
223stories submitted
5.3Kstory points
6 months agolast shared
2007people per year2026
Most-shared posts
- 1Minimum Viable Block Chain17 people · 366 points · first shared 2014 · 73 comments on HN
- 2Let's Make GitHub Better, Together13 people · 2 points · first shared 2009 · discussion on HN
- 3SVD Recommendation System in Ruby9 people · 24 points · first shared 2008 · 3 comments on HN
- 4SSTable and Log Structured Storage: LevelDB8 people · 142 points · first shared 2012 · 18 comments on HN
- 5Ruby 1.9 Internals: Ordered Hash6 people · 48 points · first shared 2009 · 11 comments on HN
- 6The "Average Page" is a myth5 people · 15 points · first shared 2016 · discussion on HN
- 7Don't "Push" Your Pull Requests5 people · 13 points · first shared 2011 · discussion on HN
- 8Latency: The New Web Performance Bottleneck5 people · 11 points · first shared 2012 · 1 comments on HN
- 9High Performance Networking in Google Chrome4 people · 285 points · first shared 2013 · 42 comments on HN
- 10Tokyo Cabinet DB: Beyond Key-Value Store4 people · 135 points · first shared 2009 · 21 comments on HN
- 11Control Groups (cgroups) for the Web?4 people · 89 points · first shared 2016 · 26 comments on HN
- 12Flow Analysis & Time-based Bloom Filters4 people · 76 points · first shared 2010 · 15 comments on HN
- 13Nginx and Memcached, an easy 400% boost in req/s4 people · 23 points · first shared 2008 · 13 comments on HN
- 14Server-Sent Event Notifications with HTML5 & Goliath4 people · 21 points · first shared 2011 · 1 comments on HN
- 15Simple SPDY and NPN Negotiation with HAProxy4 people · 14 points · first shared 2012 · 1 comments on HN
What HN people wrote when they linked it
paulgb · 2013-11-04
Ilya Grigorik ran an excellent programming blog while running PostRank (and continues to run it after being acquired by Google) http://www.igvita.com/
rubiquity · 2013-10-21
Ilya Grigorik knows his stuff incredibly well when it comes to networking. I encourage you to read his blog sometime. You'll find it very informative. Personally it is one of my favorite blogs.
http://igvita.com
subbu · 2011-06-03
One of the reasons could be its founder/CTO, Ilya Grigorik. Check out http://www.igvita.com/. He is brilliant.
dostovskk · 2017-05-27
The Minimum Viable Blockchain (https://www.igvita.com/2014/05/05/minimum-viable-block-chain/) is by far the best introduction to blockchain I have read.
jakub_g · 2016-05-12
It's far from being immediately actionable and focused on JS libs, but this book [1] from Ilya Grigorik, who is a perf engineer at Google, is a good start. It's not about JS, but a wide view, starting from TCP / HTTP(S) level, through how mobile devices work, to JavaScript.
You may want to read also some articles on perfplanet [2]
Blog of Grigorik is also intersting [3]
But as I said all those resources are about "web performance" rather than "JS performance" and are more high level overview about latencies, UX and not exactly on minimizing CPU cycles.
[1] http://chimera.labs.oreilly.com/books/1230000000545
[2] http://calendar.perfplanet.com/2015/
[3] https://www.igvita.com/
halflings · 2014-12-23
> I hate that many other languages, including C#, Ruby and Python, force me to choose between either an unordered map or a list of (key, value) tuples
Another commenter mentioned Python's OrderedDict, and Ruby's hashtables are ordered (from 1.9+):
https://www.igvita.com/2009/02/04/ruby-19-internals-ordered-hash/
It looks like C# also has an OrderedDictionary class:
http://msdn.microsoft.com/en-us/library/system.collections.specialized…
I never had to use any of these classes even if I do use hashmaps all the time, so IMHO the Python way is best (default to a 'normal' unordered hashmap, offer an ordered alternative)
discreditable · 2014-11-23
Author is going about this in completely the wrong way. The best way to determine if you should deliver webp or jpeg is to look at the browser's accept header [1]. In this header you will find a listing of what the browser is willing to accept and the order of preference.
[1] https://www.igvita.com/2013/05/01/deploying-webp-via-accept-content-ne…
konklone · 2014-11-14
> Those aren't the only downsides. The other downside is dramatically increased first byte latency due to the handshake.
The latency increased, but it doesn't have to be drastic. Ilya Grigorik has a great piece on how to get the handshake down:
https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-b…
We turn on as much of that stuff as we can for our servers, and I'm pretty happy with it.
rakoo · 2014-08-06
On the other hand, what's the problem in using TLS for cases you don't think it useful ? I can see a few reasons here, but I don't see any of them being enough not to use TLS everywhere:
- TLS is expensive: my gut says it's wrong, but I'd love to see some numbers. Ilya Grigorik [0] has done some experiments here, and I don't see TLS as really bad
- TLS is complicated: true, and we have to rely on tried and tested implementations. I'd say you'd need to do it whatever security we use (and we want security, right?)
- TLS requires certificates from the flawed CA infrastructure we have: wrong, public-key authentication isn't even the only authentication scheme possible with TLS, it's just the first one we think about (and also the most tested one).
Do you have other counter-arguments ?
[0] https://www.igvita.com/2013/12/16/optimizing-nginx-tls-time-to-first-b…
fladells · 2014-07-11
You probably mean this one:
https://www.igvita.com/2014/05/05/minimum-viable-block-chain/
If so, I agree it's a great intro to the topic.
Edit: Nvm that article is only from 3 months ago.
simonw · 2013-03-13
Chrome's networking performance optimisations are brilliant, and currently unmatched by any other browser.
http://www.igvita.com/posa/high-performance-networking-in-google-chrom…
perlpimp · 2013-02-08
there are these kinds of people for one ilya grigorik http://igvita.com
messel · 2011-10-17
Here are the ones in my reader, admittedly I haven't visited often enough over the past couple of months:
http://www.igvita.com/
http://ejohn.org/
http://codingrelic.geekhold.com/ (Denny covers assembly and networking issues often in great detail)
http://buhrmi.tumblr.com/
http://elegantcode.com/
http://72lions.com/
http://kellabyte.com/ (mobile-ish dev)
http://lambda-the-ultimate.org/
http://ihumanable.com/blog
http://4loc.wordpress.com/
http://www.maxogden.com/
http://till.klampaeckel.de/blog/
http://www.quirkey.com/blog
http://www.mikealrogers.com/
http://cocoawithlove.com/
DanielRibeiro · 2011-06-03
Congrats to Ilya. His personal blog has some of the most inpiring articles in Ruby/AI and technichal stuff in general: http://www.igvita.com
rb2k_ · 2011-04-27
The idea of lua integration reminds me of Tokyo Tyrant.
Ilya Grigorik wrote a nice article about Tokyo+Lua back in the day: http://www.igvita.com/2009/07/13/extending-tokyo-cabinet-db-with-lua/
Love the idea of server-side code
pbrumm · 2011-03-23
this has been a great set of blog posts.
since your covering all the bases. you should look at the bloom filters.
something like this could work and be created on the server.
http://dren.ch/js-bloom-filter/
(not the newest post has a compatible perl and ruby impl)
Ilya Grigorik has a couple of relevant posts
http://www.igvita.com/2008/12/27/scalable-datasets-bloom-filters-in-ru…
and
a ruby + redis backend.
https://github.com/igrigorik/bloomfilter
here are some of the memory footprints that Ilya got
* 1.0% error rate for 1M items, 10 bits/item: 2.5 mb
* 1.0% error rate for 150M items, 10 bits per item: 358.52 mb
* 0.1% error rate for 150M items, 15 bits per item: 537.33 mb
rudiger · 2011-02-18
Mr. Shaw giving an "extreme talk" on using ZeroMQ, Advanced Network Architectures With ZeroMQ, should be interesting. For those that don't know ZeroMQ and why it's so useful, here's a few links:
http://nichol.as/zeromq-an-introduction
http://www.infoq.com/news/2010/09/introduction-zero-mq
http://www.igvita.com/2010/09/03/zeromq-modern-fast-networking-stack/
http://mongrel2.org/doc/tip/docs/manual/book.wiki#x1-640005.2 <== The Mongrel2 manual is probably the best concise introduction to ZeroMQ.
mattrepl · 2010-01-21
For anyone that finds the posted site painful on the eyes, here's another introduction to SVD that is all on one page by default and isn't crowded by advertisements.
http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/
earle · 2009-11-24
tgittos · 2009-08-26
Two I have in my reader:
Gödel’s Lost Letter and P=NP - http://rjlipton.wordpress.com/
Igvata - http://www.igvita.com/ (Ruby based, sometimes too terse)
quellhorst · 2009-05-08
I would look into hosting the system at someplace like softlayer, pay special attention to the cloud offering they have coming out next week.
For file store you maybe able to save some money using http://www.danga.com/mogilefs/ instead of S3.
For your datastore checkout http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/
Also worth taking a look at is http://hadoop.apache.org/core/
chrislo · 2009-04-06
Ilya Grigorik wrote a great article on recommendation systems (with code in Ruby):
http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/
abijlani · 2008-11-10
Here's a great hpricot tutorial
http://www.igvita.com/2007/02/04/ruby-screen-scraper-in-60-seconds/
gunderson · 2008-07-03
You don't have to do anything too sophisticated to get some great results for your users.
Check out this article and collaborative filtering library:
http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/
I actually want to play with this more, so if anyone wants to create a google group to share implementation approaches, etc., just reply to this and I'll create it.
cstrahan · 2023-12-10
JNI allows one to write C code that bridges the gap between the Java VM and existing C libraries. That C code must be compiled, and must follow some rather tedious patterns (regarding method signatures and other little details). It’s a bit of a pain to write and compile, if all you want to do is quickly call some function from an existing C library.
Also, because symbol resolution is done at compile-time, this is a non starter for something like JRuby, wherein one would want to write plain Ruby code to call a C function — rather than writing Ruby, Java and JNI flavored C code. Ruby is an interpreted language, so calling into or out of C requires dynamic symbol resolution, and for the C->Ruby callback case, a closure must be created on the fly (that is, a C ABI function must be JIT compiled for that method or proc). This isn’t particular to Ruby — any interpreted language will run into this issue on the JVM (picture Python and its ctypes and/or cffi libraries).
You can see some of the hoops that JRuby had to jump through to support this use case, in order to provide a JRuby implementation of the ‘ffi’ gem interface:
https://javadoc.io/doc/org.jruby/jruby-core/9.2.13.0/org/jruby/ext/ffi…
https://www.igvita.com/2009/01/15/bridging-mri-jruby-rubinius-with-ffi…
It’s been a minute, so my recollection may be wrong, but IIRC JRuby uses Java and JNI to call into a C shim that then calls into libffi to handle things like creating C ABI conforming closures and such on the…
hkon · 2016-10-28
I like this one.
https://www.igvita.com/
manigandham · 2016-07-16
> Unless you control the TLS terminators
Nope, this is all about trusting vendors. Unless you're building your own computer chips and laying your own fiber, you're just blaming a vendor for what seems like personal issues.
> problem is generally not in the network latency
Again years of research has shown this is one of the biggest issues on the web. Read https://www.igvita.com/ for some good info.
> Yes, both technically use reverse proxying technology in their stack, but in the end they are completely different approaches.
Nope, this clearly shows you don't have any understanding. CF is a CDN. That's it. Instead of like other CDNs that offer a CNAME that you point your DNS to, they just offer a free DNS service that automates the CNAME entry into a single button.
All CDNs today are reverse proxies that will either return a cached response or proxy the request to the origin if the cache is empty. You can also use every CDN to proxy dynamic requests and many do. And yes, there are a few that offer a push model where you upload assets explicitly but this is the exception, not the standard.
You're talking about CF as if they're something unique when they do exactly the same thing as everyone else. Use them on a subdomain if you want and leave the main domain unproxied, use them only for static files or for dynamic requests, use them to proxy everything or cache everything including HTML. It's all just configuration. Why is this such a problem for you?
Smerity · 2012-03-26
Antirez mentions that a limited set of on-disk data structures could work well but that it will take one or two years to even reach the drawing board. Fair enough -- Redis is first and foremost an in-memory database.
If there was the time though, I'd love to see LevelDB[1] merge the gap between in-memory and on-disk. Inspired by Google's BigTable, all keys are kept sorted on disk (see: SSTable[2]). Keys, lists, sets, sorted sets and hashtables could be encoded in sorted key-value form and would be reasonably (though not tremendously) efficient to retrieve, especially if the query can be converted to a range query (i.e. list retrieval or set intersection). Keep hot data in memory, the rest ends up securely on disk.
Yes, Cassandra is based on the same lineage but it's not simple or clean to operate. Redis is a pain free setup, features a simple API but has no transparent way to overflow to disk for cold data. LevelDB is an optional backend for Riak but I must admit I've not explored Riak heavily... Have I missed a contender from another database crowd?
[1]: http://code.google.com/p/leveldb/
[2]: http://www.igvita.com/2012/02/06/sstable-and-log-structured-storage-le…
stevenbedrick · 2010-10-22
Sorry for the delay, I just now saw that somebody had replied. Regarding links to the systems- the only one that was ever publicly accessible was for a Facebook app that I wrote some years back that was let researchers post links to their publications on their profiles, and also featured a recommendation system to find other users of the app that had "similar" publications. It's unfortunately fallen into a pretty sorry state of disrepair, as I haven't had any time to devote to maintenance in more than two years and Facebook's APIs have changed quite a bit since then. I described the underlying system a little bit in a conference paper: http://view.ncbi.nlm.nih.gov/pubmed/18999247
The other systems I've used the approach for have all been along either bibliometric/bibliographic lines, or have been relating to content-based image retrieval. It's a pretty robust approach, but can take a little bit of tuning to get just right- coming up with a good evaluation strategy is important to getting the most out of it, I've found.
As far as references that were useful:
Ilya Grigorik has a very accessible getting-your-feet-wet tutorial on his site: http://www.igvita.com/2007/01/15/svd-recommendation-system-in-ruby/
It might be a little dated w.r.t. specific libraries or APIs, but the basic technique is there. For a more comprehensive look at the SVD-IR approach, take a look at:
Berry et al. Using Linear Algebra for Intelligent Information Retrieval. SIAM Review (1995) vol. 37 (4)…
jfornear · 2009-06-05
2nd igrigorik http://www.igvita.com/