Felipe Hoffa
medium.com/@hoffa · Personal blog · on HN since 2015
Felipe Hoffa, a Google developer advocate, uses BigQuery to analyze large public datasets like GitHub, Hacker News, and Reddit.
What HN saysHN commenters cite his tabs-versus-spaces analysis of GitHub files and his Hacker News and Reddit datasets on BigQuery; he also appears answering questions on dataset updates.
29HN people shared it
6linked it in comments
42stories submitted
319story points
6 years agolast shared
2015people per year2026
Most-shared posts
- 1Tabs or spaces – Parsing a 1B files among 14 programming languages10 people · 130 points · first shared 2016 · 122 comments on HN
- 2Hacker News on BigQuery: Now with daily updates. Top domains and time to post?4 people · 57 points · first shared 2017 · 9 comments on HN
- 3All the open source code in GitHub now shared in BigQuery: Analyze all the code3 people · 16 points · first shared 2016 · discussion on HN
- 4The top weekend languages – this time according to GitHub commits3 people · 9 points · first shared 2017 · 1 comments on HN
- 5The top GitHub projects per country2 people · 11 points · first shared 2017 · 1 comments on HN
- 6What countries have more open source developers per capita than the US? – Medium2 people · 5 points · first shared 2016 · discussion on HN
- 7The news sources that Reddit prefers2 people · 3 points · first shared 2017 · 2 comments on HN
- 8The most famous Reddit accounts2 people · 3 points · first shared 2017 · discussion on HN
- 9GitHub Archive fully updated in BigQuery2 people · 2 points · first shared 2016 · discussion on HN
- 10Counting uniques faster in BigQuery with HyperLogLog++1 people · 20 points · first shared 2017 · discussion on HN
- 11The top contributors to GitHub – 20171 people · 14 points · first shared 2017 · discussion on HN
- 12Reddit’s uptime (2008–2016)1 people · 13 points · first shared 2016 · discussion on HN
- 13Winning arguments with data1 people · 9 points · first shared 2017 · discussion on HN
- 14Using Wikidata+Wikipedia+BigQuery to visualize the DNC and RNC 2016 conventions1 people · 4 points · first shared 2016 · discussion on HN
- 15Real people don’t use UTM codes1 people · 3 points · first shared 2020 · discussion on HN
What HN people wrote when they linked it
rhacker · 2019-08-16
Wouldn't this make sense?
https://medium.com/@hoffa/400-000-github-repositories-1-billion-files-…
So the number of people that prefer spaces far outweighs the tabbers.
Now, take the pool of people that use spaces and tabs and let them compete for a 100k job. 9 times out of 10 even if randomly hired it will be a space person.
Also, as a by the way, on the medium post. The reason Java has so much tabs is specifically due to the default Eclipse formatting that prefers a mix Tab/Space approach. Which is absolutely horrifying in my personal opinion. I don't have any numbers but Eclipse probably holds 80%, followed by Netbeans and/or IntelliJ. (both of which default to spaces)
subroutine · 2017-08-31
I recently wrote a little Python script to scrape the url from top submissions to Reddit's /r/The_Donald vs. /r/EnoughTrumpSpam to get an idea of the most conservative-leaning and liberal-leaning news outlets (e.g. more specifically which news sites conservative-leaning and liberal-leaning redditors use to support their respective narratives). Here are the results:
https://i.redd.it/45ezt3j9hziy.png
Edit: Oh and I almost forgot (and to bring this full circle) a Googler saw my post about this and made an interactive version:
https://medium.com/@hoffa/reddit-favorite-sources-the-most-linked-site…
Edit: and to really bring this full circle, this Googler made an interactive version about HN:
https://medium.com/@hoffa/hacker-news-on-bigquery-now-with-daily-updat…
fhoffa · 2019-03-04
Hi, Felipe Hoffa at Google here.
We're aware the dataset hasn't been updated since a month ago, and we are working to fix it. You can track the issue here:
- https://issuetracker.google.com/issues/127132286
In the meantime you can still play with the dataset, and dig into the full history of Hacker News - less this last month. I left some interesting queries to get you started here:
- https://medium.com/@hoffa/hacker-news-on-bigquery-now-with-daily-updat…
Confusion · 2017-06-08
By popular vote: spaces. See https://medium.com/@hoffa/400-000-github-repositories-1-billion-files-… Previous discussion: https://news.ycombinator.com/item?id=12397246
minimaxir · 2016-06-29
The GitHub Archive dataset was updated as well. Example BigQuery to get the Top Repositories from 2015-2016 YTD, by the number of Stars given during that time:
SELECT repo.id, repo.name, COUNT(*) as num_stars
FROM TABLE_DATE_RANGE([githubarchive:day.], TIMESTAMP('2015-01-01'), TIMESTAMP('2016-12-31'))
WHERE type = "WatchEvent"
GROUP BY repo.id, repo.name
ORDER BY num_stars DESC
LIMIT 1000
Which results in this output: https://docs.google.com/spreadsheets/d/16yDS2wDdDOTxjVsjGvWmpHVsOIU65w…
Since the query only hits 3 columns, it only uses 15.4GB of data (out of a 1TB allowance)
More information on the GitHub Archive changes: https://medium.com/@hoffa/github-archive-fully-updated-notice-some-bre…
kbenson · 2017-09-25
I believe the number of voters in key districts needed to swing the electoral college to Clinton was in the tens of thousands. If it was targeted perfectly (which of course is impossible), that's almost two dollars a person that needed to be influenced. How many Facebook ads impressions does that buy per person? While I think it's unlikely that anyone could have accurately targeted to that degree, it does put into perspective just how important targeted advertising might be to something like this.
Additionally (and I think more likely), the advertising might have been been used more as an agitation device, and to reduce the trust in traditional information through highly effective misinformation. If you can start viral memes, it doesn't matter how many impressions you paid for, what's important is how far the meme spreads in the end. A single shared article can lead to tens of millions of views and re-shares. In this case, money is purely a way to help get critical mass for your specially crafted meme, so it doesn't necessarily compare to candidate spending directly, which needs to both educate and counteract opponent messaging. Targeted disinformation can just ignore failed meme campaigns and counteractions to them and move onto the next meme, since it's effectively anonymous.
1: https://medium.com/@hoffa/hillary-only-needed-to-switch-53-650-voters-…