Pinterest Engineering
engineering.pinterest.com · Company blog · on HN since 2012
Pinterest's engineering team writes about its infrastructure, open-source releases, and front-end and backend migrations.
What HN saysHN commenters cite it as a good example of a company engineering blog, including its article on switching template rendering, and Pinterest's Elixir tools come up.
95HN people shared it
24linked it in comments
112stories submitted
1.8Kstory points
10 years agolast shared
2012people per year2026
Most-shared posts
- 1Introducing new open-source tools for the Elixir community6 people · 53 points · first shared 2015 · 1 comments on HN
- 2Migrating Pinterest profiles to React4 people · 107 points · first shared 2016 · 43 comments on HN
- 3Http://engineering.pinterest.com/4 people · 1 points · first shared 2014 · discussion on HN
- 4Memcache games4 people · first shared 2012
- 5Open-sourcing Teletraan, a code deployment system3 people · 80 points · first shared 2016 · 16 comments on HN
- 6Open-sourcing Terrapin: A serving system for batch generated data3 people · 18 points · first shared 2015 · discussion on HN
- 7Zookeeper resilience at Pinterest3 people · 10 points · first shared 2014 · 2 comments on HN
- 8Building Pinterest's follower model from scratch3 people · 8 points · first shared 2013 · discussion on HN
- 9Monitoring A/B experiments in real-time3 people · 6 points · first shared 2016 · discussion on HN
- 10Open-sourcing Rocksplicator, a real-time RocksDB data replicator3 people · 5 points · first shared 2016 · discussion on HN
- 11Demystifying SEO with experiments2 people · 200 points · first shared 2015 · 51 comments on HN
- 12Sharding Pinterest: How we scaled our MySQL fleet2 people · 151 points · first shared 2015 · 85 comments on HN
- 13How we switched our template rendering engine to React2 people · 122 points · first shared 2016 · 39 comments on HN
- 14Open-sourcing Pinterest MySQL management tools2 people · 101 points · first shared 2015 · 8 comments on HN
- 15Scalable A/B Experiments at Pinterest2 people · 62 points · first shared 2014 · 15 comments on HN
What HN people wrote when they linked it
pinterestjobs · 2014-04-18
San Francisco - Pinterest
Pinterest is a visual discovery tool for collecting and organizing things you love and enjoy. Millions of people use Pinterest daily in their lives and work. No matter what you're interested in, there's a place for it here.
As a Software Engineer on the Growth Team at Pinterest, you’ll have a huge impact on how we design, build and deliver our service to millions of people. You should be a generalist who enjoys working across the software stack with a strong focus on data and a keen eye for product. Prototyping, running A/B tests, pushing code into production, weighing in on the best ways to grow Pinterest and engage Pinners...if that all sounds good to you, then this might just be the place for you.
Responsibilities:
-Find ways to get new users on the site and engaged with the product. This includes SEO, signup optimizations, new-user experiences and any product change you think needs to happen. For more about how we work, read all about how we recently rebuilt our new user experience and created our SEO sitemaps on our engineering blog: http://engineering.pinterest.com/
-Be heavily involved in every step of the product development process, from ideation to implementation to release.
-Build new features that increase social sharing both within Pinterest and externally with friends (SMS, Facebook, Twitter).
-Improve the messaging platform so that we can send our Pinners engaging and timely notifications and emails.
-Run experiments on new and…
djm_ · 2017-01-12
It's possible but 'quite easy' does it a disservice as it is often not the case at any reasonable scale. Pinterest wrote a great article [1] on how they've handled it recently.
[1] https://engineering.pinterest.com/blog/how-we-switched-our-template-re…
rozap · 2016-11-10
two things:
1) elixir is relatively young so there aren't many projects that have had a chance to grow to be huge. so yea, take everything you read about it with that grain of salt. people haven't grown to hate it yet...maybe everyone will always love it, but I wouldn't bet on it.
2) when used in certain contexts, elixir applications may end up being quite a bit smaller than comparable java/scala/python/whatever apps. A lot of what we end up writing with microservices is just basic RPC and fault tolerance boilerplate stuff, which OTP takes care of out of the box. Example, pinterest re-writing a 10kloc java app in 1kloc of elixir, running it on fewer servers: https://engineering.pinterest.com/blog/introducing-new-open-source-too…
dv_says · 2016-01-01
Using Elixir here for a couple of months, for an app backend, and even for shell script type of projects. Great language, no surprises, and very responsive community -- really, my biggest wish is for more people to try it, as I think it's still relatively niche. Had no previous Erlang experience. While you don't have to write Erlang code itself, you'll over time become familiar with Erlang/OTP architecture concepts, such as GenServer, ETS, distributed nodes, and so on.
Was very happy to see this recent post from Pinterest [1] as a sign that some of the more "mainstream" teams are starting to look at it.
Also, a repost [2] of some helpful tips if you're just getting started.
[1] https://engineering.pinterest.com/blog/introducing-new-open-source-too…
[2] https://news.ycombinator.com/item?id=10278870
SEJeff · 2015-05-19
And Esty: https://codeascraft.com/
Netflix: http://techblog.netflix.com/
Facebook: https://code.facebook.com/posts/
Instagram: https://instagram-engineering.tumblr.com/
Pinterest: http://engineering.pinterest.com/
These are all excellent blogs to keep up on in your free time.
kiyoto · 2013-02-23
I don't mean frontend but anything related to web services, really. "Lots of http transport stuff" is still "web programming" (for me, at least).
>Msgpack just doesn't seem, to me, as a really great fit for anything in particular.
Pinterest disagrees: http://engineering.pinterest.com/posts/2012/memcache-games/
To give you some context, frsyuki developed MessagePack originally for RPC (he was building a distributed key-value store in college) and made a conscious decision not to add a string type for maximal cross-language compatibility. Like a lot of other technology, MessagePack has outlived its original purpose, and there is a lot of demand now for it to support the (utf-8) string type, not just byte array.
xfactor973 · 2016-08-18
After checking out the blog post https://engineering.pinterest.com/blog/tracker-ingesting-mysql-data-sc… I thought it would be interesting to see what happens if I converted their NullEscape.c program to Rust. The results were encouraging so I submitted a PR :)
dmuth · 2016-08-02
What indeed. Having that many tables in a database is just asking for trouble.
My gut feeling for a database of this size is to suggest sharding. While I don't claim to be an expert in this sort of thing, here's how Pintrest approached a similar problem: https://engineering.pinterest.com/blog/sharding-pinterest-how-we-scale…
toomuchtodo · 2016-04-08
Pinterest as well uses manual intervention:
https://engineering.pinterest.com/blog/open-sourcing-pinterest-mysql-m…
pmarreck · 2016-03-01
Elixir @ Pinterest, for anyone who missed it
https://engineering.pinterest.com/blog/introducing-new-open-source-too…
Also, just noticed that a relatively new but nice-seeming screen sharing app called CrankWheel put out a talk covering their realtime Erlang/Elixir-backed architecture: https://www.youtube.com/watch?v=6RGCPkdJBjI
dang · 2016-02-12
Thanks! We changed the URL to that from https://engineering.pinterest.com/blog/under-hood-teletraan-deploy-sys…
Xeroday · 2016-02-12
Here's the latest blog post, which includes the link to the code: https://engineering.pinterest.com/blog/open-sourcing-teletraan-code-de…
nikolay · 2016-02-12
merdreubu · 2016-02-01
I know about Bleacher Report[1],the second biggest sport site in the world after ESPN, and Pinterest[2].
[1]https://vimeo.com/128349620
[2]https://engineering.pinterest.com/blog/introducing-new-open-source-too…
bandris · 2015-12-19
Using half as many servers as before. More info on the blog:
https://engineering.pinterest.com/blog/introducing-new-open-source-too…
kevinbowman · 2015-09-14
The URL https://engineering.pinterest.com/blog/open-sourcing-terrapin-serving-… gives more info, which this article links through to.
detaro · 2015-07-31
shackenberg · 2015-06-01
For reference, here is the link to the real arxiv page: http://arxiv.org/abs/1505.07647 and here is the blog post announcing the paper: http://engineering.pinterest.com/post/120111908004/building-a-scalable…
mattj · 2014-06-19
A teammate of mine wrote a post about our redshift setup a few months ago with some more details: http://engineering.pinterest.com/post/75186894499/powering-interactive…
shykes · 2013-03-07
Here's one possible answer, for one specific use case: http://engineering.pinterest.com/posts/2012/memcache-games/
kzk_mover · 2012-12-29
For those who are interested in Pinterest usecase (Memcached + MessagePack), please access to this url too.
> http://engineering.pinterest.com/posts/2012/memcache-games/
visualsearchsv · 2016-01-28
Inspired by the Pinterest paper at KDD on implementing Visual Search, I have created this barebones but functional implementation of a Visual Search server using ~450,000 Female Fashion images crawled from an image aggregation website. The code uses Pool 3 layer features extracted from the Google's latest Inception model using Tensor Flow. The extracted vectors are then indexed using an Approximate Nearest Neighbor implementation from Nearpy. The AMI provided contains both images and pre-computed vectors.
In future I plan to add more images ~2 Million in the same domain. Test various combination of nearest neighbor indexes and multiple vectors per images using some form of a multibox style detector. I will also add a script to launch spot GPU instances via Cloudformation to economically index images using S3 and SQS. I am building a companion iOS swift app however, since Tensor Flow hasn't been ported to iOS yet, its still in development.
https://engineering.pinterest.com/blog/building-scalable-machine-visio…
pdevr · 2015-05-19
These are some of the top sites' engineering blogs which I collected over a period of time - I removed those which have already been posted:
Google: http://googledevelopers.blogspot.com/ http://google-engtools.blogspot.com/ http://googleresearch.blogspot.com/
YouTube: http://youtube-eng.blogspot.com/
Baidu: http://usa.baidu.com/category/baidutechblog/
Yahoo: http://yahooeng.tumblr.com/ http://yahoodevelopers.tumblr.com/
Wikipedia (closest I could find): https://blog.wikimedia.org/c/technology/
Amazon: https://aws.amazon.com/blogs/aws/
Twitter: https://engineering.twitter.com/
eBay: http://www.ebaytechblog.com/
Microsoft: http://microsoftengineering.com
FourSquare: http://engineering.foursquare.com/
Apple (closest): https://developer.apple.com/
Paypal: https://www.paypal-engineering.com/ https://devblog.paypal.com/
Dropbox: https://blogs.dropbox.com/tech/
Edit: More (collected from this link: http://www.quora.com/What-are-the-top-startup-engineering-blogs):
Yelp: http://engineeringblog.yelp.com/
Quora: http://engineering.quora.com/
Yammer: http://eng.yammer.com/blog/
AirBnB: http://nerds.airbnb.com/
Square: https://corner.squareup.com/
Flickr: http://code.flickr.net/
Soundcloud: https://developers.soundcloud.com/blog/
Cloudera: http://blog.cloudera.com/blog/
Slideshare: http://engineering.slideshare.net/
Heroku: http://engineering.heroku.com/
Pinterest: http://engineering.pinterest.com/
frsyuki · 2012-06-11
Although the original blog post focuses on JavaScript and browsers, MessagePack itself doesn't mainly focus on them.
A major use case of MessagePack is to store serialized objects in memcached. A blog post written by Pinterest describes this use case (http://engineering.pinterest.com/posts/2012/memcache-games/).
They use MessagePack with Python which is faster than one with JavaScript. They could store more objects in a server without performance declination (e.g. gzip).
It's true that MessagePack is not always faster than JSON (e.g. within browsers), and it's not always smaller than other serialization methods (e.g. with gzip compression). So we should consider that which serialization methods should I use for "my" case.
There are also general tendency which is helpful to select MessagePack or JSON:
MessagePack is faster to serialize binary data such as thumbnail images.
MessagePack is better to reduce overheads to exchange small objects between servers.
JSON is better to use it with browsers.