NYT Open
open.blogs.nytimes.com · Company blog · on HN since 2007
The New York Times engineering team writes about its CMS, HTTPS migration, recipe data extraction, and open source web tools.
What HN saysCommenters call the open blog interesting and point to it as evidence the NYT has strong web developers, citing its recipe-extraction piece on conditional random fields.
247HN people shared it
82linked it in comments
266stories submitted
7.1Kstory points
9 months agolast shared
2007people per year2026
Most-shared posts
- 1Extracting Structured Data from Recipes Using Conditional Random Fields (2015)9 people · 248 points · first shared 2015 · 20 comments on HN
- 2Homepage6 people · first shared 2010
- 3The New York Times Is Now Available as a Tor Onion Service5 people · 1.1K points · first shared 2017 · 192 comments on HN
- 4Building a Text Editor for a Digital-First Newsroom5 people · 486 points · first shared 2018 · 168 comments on HN
- 5NYTimes Opensources Their Deep Linking JS5 people · 174 points · first shared 2011 · 50 comments on HN
- 6Embracing HTTPS5 people · 98 points · first shared 2014 · 52 comments on HN
- 7NYTimes uses Hadoop, S3, EC2, and some custom code to handle PDF generation for 4TB worth of data5 people · 37 points · first shared 2007 · 4 comments on HN
- 8Homepage5 people · first shared 2010
- 9The New York Times course to teach its reporters data skills is now open-source4 people · 427 points · first shared 2019 · 82 comments on HN
- 10React, Relay and GraphQL: Under the Hood of the Times Website Redesign4 people · 198 points · first shared 2017 · 112 comments on HN
- 11Scoop: A Glimpse Into the NYTimes CMS4 people · 192 points · first shared 2014 · 71 comments on HN
- 12Remembering a Programming Language That Helped Shape the Digital New York Times4 people · 129 points · first shared 2019 · 44 comments on HN
- 13Announcing the New York Times Congress API4 people · 60 points · first shared 2009 · 10 comments on HN
- 14Library: A Collaborative Documentation Site4 people · 51 points · first shared 2019 · 7 comments on HN
- 15NYTimes – How to Grow as an Engineer (Working Remotely)4 people · 18 points · first shared 2018 · 1 comments on HN
What HN people wrote when they linked it
wildpeaks · 2017-03-01
I recall the creator of the open source library D3.js also used to work there and did a lot great interactive pieces for articles.
Their open source blog is also quite interesting: https://open.blogs.nytimes.com
wiremine · 2014-03-20
And the NY Times has done a great job:
http://open.blogs.nytimes.com/
That said, I think those are exceptions, not the rule.
lchang · 2013-03-14
Yes, but the New York Times employs top-notch webdevs.
See http://open.blogs.nytimes.com/ and http://www.nytimes.com/projects/2012/snow-fall/
So they have the skills, but maybe it's for other reasons they don't follow those practices.
donohoe · 2010-06-12
Ok, where to begin...
- "The Times has tried twice to create pay-wall models that have utterly failed"
Um, not true. They did it once. It was called Times Select. It actually worked quite well, however the advertising market picked up in a big way where revenue from ads would exceed subscriptions. Not the case anymore, but thats another story.
- "The Times does very little investigative journalism of significance"
Ok, I'm gonna leave this one alone. Clearly we are on different planets.
- "The Times has strong pretensions, as evidenced by its coverage of socialite weddings, etc."
I guess so... ? Is it? I mean its not tabloid, but I think thats good, right? Whats the "etc"?
- "In the buildup to the Iraq war it was the Times' sloppy fact checking that sold the war to many, many people who had been skeptical prior to reading its coverage."
I'm not sure how this relates to helping your point about "transition into the digital age" or the whole "tweet" thing.
- The Times attack on the Pulse news reader turned what should have been superb publicity into a petty attack on a small startup that obviously loves the Times.
Yeah, I personally think that was short-sighted.
"For all its flaws, it's still a great paper, but far less great than it once was or than it could be if it had more sense and guts and had to fight harder to maintain its position."
Been waiting to say this... ahem..."{citation needed}"
"Banning a word like "tweet" suggests…"
Okay, I think you need to…
larsonian · 2026-02-04
NYT is more of a tech company than you might think [1] and they've been one for longer than you might think: the de-facto standard profiler for Perl [2], of all things, comes from them.
[1] https://open.nytimes.com/
[2] https://metacpan.org/pod/Devel::NYTProf
manvillej · 2024-11-04
I have a colleague in tech over there on the product side. They do a ton of very interesting work: https://open.nytimes.com/
They have a much higher bar for quality and boutique solutions than almost any organizations.
shagie · 2022-10-04
https://open.nytimes.com/a-new-way-to-share-new-york-times-stories-3e6…
Linking to the site directly, likely with a bit of tracking information related to me, but allowing non-subscribers to access it without hitting the paywall. It also gives the actual experience.
For example - from https://news.ycombinator.com/item?id=32849962
Compare: https://archive.ph/OJYqz
and https://www.nytimes.com/interactive/2022/09/14/climate/san-francisco-f…
jobu · 2022-02-02
Data-journalism is actually a thing, and bigger organizations like the New York Times have done some pretty impressive work building tools and training programs for their reporters.
https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spr…
Terretta · 2019-10-07
Times Open — Sharing our stories of making digital products at The New York Times.
https://open.nytimes.com
cyborgx7 · 2019-06-13
"Open source" is what this sub-blogspam level article chose to call it. The NYT called it "Releasing our materials".
https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spr…
maccard · 2019-06-13
Any reason this links to a non-original source, with one of those annoying full screen email grab modals, when the article contains a link to the original release [0] ?
[0] https://open.nytimes.com/how-we-helped-our-reporters-learn-to-love-spr…
amplecooz · 2019-05-07
This article from the NYT tech blog is a really good read on how they created a custom text editor for the newsroom: https://open.nytimes.com/building-a-text-editor-for-a-digital-first-ne…
They built their custom app with the open source ProseMirror (http://prosemirror.net/) toolkit.
mlthoughts2018 · 2018-06-23
Actually it looks like this is the one I was responding to: < https://open.blogs.nytimes.com/2015/08/11/building-the-next-new-york-t… >.
timClicks · 2018-05-16
If you have sufficient labelled data, conditional random fields works well for this kind of problem. A technical team from NY Times have a great piece on it https://open.blogs.nytimes.com/2015/04/09/extracting-structured-data-f…
eggpy · 2017-12-29
In another blog post they inform us that they have been ranked a Top Company for Women in Technology[0]. I'm not familiar with the Anita Borg Institute that did the ranking, but I think it's great to see anyways.
[0] - https://open.nytimes.com/the-new-york-times-ranks-as-a-top-company-for…
exogen · 2017-10-26
Please, tell me more about how you’ve profiled both the performance of my apps and my development productivity! Amazing of you to provide this for me.
Meanwhile, NYTimes:
> “Facebook developed it to provide a data source that can evolve without breaking existing code and to favor speed on low-powered and low-quality mobile devices.”
[1] https://open.nytimes.com/react-relay-and-graphql-under-the-hood-of-the…
jbochi · 2017-10-02
Very interesting. I've implemented something similar[1] using a pure Collaborative Filtering approach[2][3], that I think works better for me, but it's unable to recommend unpopular repositories.
The New York Times recommender system uses a hybrid approach (Content Based + Collaborative Filtering) called Collaborative Topic Modeling on top of LDA[4]. It would be interesting to try that.
[1]: https://github-recs.appspot.com/
[2]: https://medium.com/towards-data-science/recommending-github-repositori…
[3]: https://github.com/jbochi/facts
[4]: https://open.blogs.nytimes.com/2015/08/11/building-the-next-new-york-t…
awhitty · 2015-08-23
They also have great write-up detailing their use of CRFs to extract structured data from recipes.
http://open.blogs.nytimes.com/2015/04/09/extracting-structured-data-fr…
jprob · 2015-02-12
We've been using Go in production on our proprietary email platform for a couple years now for daemon services, web APIs and EMR jobs[1]. A few other teams have also picked up the language and are building out new systems to support comments, mobile and I think a maybe a few other pieces of the site.
Go is definitely not our only tool in the box and many teams (including mine) still use Python, C++ or whatever works best for the job at hand, but I think there definitely is a trend of Go being the choice for a lot of new server side systems.
[1] http://open.blogs.nytimes.com/2014/07/10/emr-streaming-in-go/
danso · 2013-01-29
To say that the NYT is the most tech progressive media entity is understating it...besides their vast array of web and mobile apps and forays into new story telling methods (the Avalanche story was a hit here)...they've hired tech stars such as Jeremy Ashkenas (CoffeeScript/Bacbkone) and Mike Bostock (D3) whose work benefits the online ecosystem as a whole, not just the media world, and they pour a fair amount of resources into open source projects and news data APIs:
http://open.blogs.nytimes.com/
Of course it's to be expected that they have one of the more prominent digital departments among media companies, given their size. But I think it's a fair to argue they've gone way beyond just trying to put out a decent digital media project, just as other notable digital companies innovate and produce and share beyond their core selling products.
mtlynch · 2021-10-26
>Split up ingredients into more parts.
I wrote a service called Zestful that does exactly this.[0]
It's an interesting problem. It's hard to train an ML system on the distinction between a preparation instruction like "finely chopped" and a note like, "just the white parts." My system treats "just the white parts" as a preparation instruction, but it throws away as irrelevant things like, "My favorite brand of black pepper is Smith's Fine Peppers." But I've looked at thousands of ingredients, and there are lots of wacky edge cases.
I created the service by adapting an open source project the NY Times published and then abandoned.[1] They were trying to index all of their old recipes for what would eventually become cooking.nytimes.com. Their repo felt very experimental and half-baked, so it was interesting to turn it into a production service. I wrote about the process of creating the service on my blog.[2]
[0] https://zestfuldata.com/demo
[1] https://open.blogs.nytimes.com/2015/04/09/extracting-structured-data-f…
[2] https://mtlynch.io/resurrecting-1/
qrohlf · 2019-04-02
I've been following this space for some time [1], and for me the constant struggle has been finding something that is both batteries-included, so I don't have to build a usable editor from primitives on projects where the core differentiator is not related to how content gets authored, and mobile-friendly, so that I don't have weird capslock bugs on iOS [2], completely broken Android support [3], and just general mobile weirdness [4].
To the best of my knowledge, Prosemirror has been most willing to prioritize mobile support, and currently has the best mobile browser coverage of the three most commonly mentioned modern contentEditable options (prosemirror, draft-js, quill). Unfortunately, building an editor with Prosemirror is a very hands-on process that requires you to make all the UI decisions and most of the UX decisions yourself, which isn't a great fit for most projects that don't need to innovate in the actual content creation side of things.
I would LOVE it if a batteries-included editor like this, which clearly had a lot of thought put into the UX and UI aspects, was built on the very-robust technical foundation of Prosemirror (or similar). There are a few efforts at this on GitHub, but nothing so far with more than one or two contributors that I've found. The backend of editor.js looks solid, but without any discussion of mobile IME support, and without the backing of big players like NYT [5], realistically I can only guess that mobile support for editor.js will…
git-pull · 2018-02-22
This may not be what you're looking for, but Google seems to prefer Medium content.
I think it could be natural. A lot of smart interlinking of posts, clustering different niches, and posts being spread across social websites.
I can even give a direct example. My book The Tao of tmux is available for free online. It used to be in the google top 10 for tmux. Now, a newer article on HackerNoon is up at the very top: https://hackernoon.com/a-gentle-introduction-to-tmux-8d784c404340
I think people just like the Medium posts, heh. They made a publishing platform that attracted talented writers, and the feedback loop with readers seems to be encouraging it to grow.
Medium actually hampers my personal efforts since I keep my own static website and wrote a custom blog-like application I post. I feel I may end up having to move writing efforts there since the velocity there is so big.
P.S. NYTimes digital team uses Medium: https://open.nytimes.com/ I think that demonstrates to me people just like the product and it's going to self-perpetuate until someone devises something better.
macNchz · 2025-06-07
The NYT does have a direct-sold ads business and first-party data platform for targeting them: https://open.nytimes.com/to-serve-better-ads-we-built-our-own-data-pro…
xyst · 2024-08-20
Onion site? As in, there’s a mirror on tor?
(Edit) TIL: https://open.nytimes.com/https-open-nytimes-com-the-new-york-times-as-…
kastnerkyle · 2023-10-27
There was a really nice post on doing this kind of thing with CRF back in 2015 [0]. Open source data, and code on github. Also a nice tutorial on structured prediction using CRF type models.
Would be interesting if you could prompt, LoRA distill, or use modern LLM tricks against a well-labeled and curated set, similar to how other tagging problems are handled with modern pretrained models.
[0] https://open.nytimes.com/our-tagged-ingredients-data-is-now-on-github-…
reaperducer · 2023-07-31
If that's what you're into, the New York Times has been on Tor Onion for six years:
https://www.nytimesn7cgmftshazwhfgzm37qxb44r64ytbb2dj3x62d2lljsciiyd.o…
https://open.nytimes.com/https-open-nytimes-com-the-new-york-times-as-…
usrusr · 2023-07-29
According to the New York Times, no one noticed: https://open.nytimes.com/we-re-launched-the-new-york-times-paywall-and…
That cut looks a lot like a consequence of changing approaches to monetarization.
philshem · 2023-07-29
Interesting. They had a major refactor of their paywall in August 2019. Maybe that turns people off to posting content that isn’t unique to the NYT. Or maybe paywalled articles are less likely to make the HN front page (?).
https://open.nytimes.com/we-re-launched-the-new-york-times-paywall-and…
mutant · 2023-02-01
It could be an employee pet project. Kinda like what happened here
https://open.nytimes.com/we-built-a-collaborative-documentation-site-d…