Flickr Code Blog

This is the best way, and actually the only sensible way to approach the problem. I first read about it here https://code.flickr.net/2010/02/08/ticket-servers-distributed-unique-p…

I had asked a similar question on Quora [1] few years back. Here is the summary: - Facebook: https://www.facebook.com/Engineering?sk=notes - Google: http://google-engtools.blogspot.com/ - Twitter: http://engineering.twitter.com/ - Linkedin: http://engineering.linkedin.com/blog - Dropbox: https://tech.dropbox.com/ - Instagram: http://instagram-engineering.tumblr.com/ - Netflix: http://techblog.netflix.com/ - Flickr: http://code.flickr.com - Etsy: http://codeascraft.etsy.com/ - Yelp: http://engineeringblog.yelp.com/ - Heroku: https://blog.heroku.com/ - Airbnb: http://nerds.airbnb.com/ - Bitly: http://word.bitly.com/ - Evernote: https://blog.evernote.com/tech/ - Highscalability: http://highscalability.com/ [1] https://www.quora.com/What-are-the-companies-that-have-some-of-the-bes…

Obligatory follow-up link to that xkcd: http://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Hello, Sentinel 2 is doing quite a good work for many users, here is for example Flickr report: http://code.flickr.net/2014/07/31/redis-sentinel-at-flickr/ Of course Sentinel does not make Redis conceptually different from what it is from the point of view of consistency guarantees during failures. It performs best-effort attempt to select the best slave to retain writes, but under certain failure modes its possible to lose writes during a failover. This is common with many failover solutions of *SQL systems as well btw. It depends on your use case if this is an affordable risk or not. For most Redis use cases, usually the risk of losing some writes after certain failovers is not a big issue. For other use cases it is, and a store that retains the writes during all the failure scenarios should be used.

Excellent writeup! I'm amazed people signed up to play their test game. Does anyone know if the points you get translate to any sort of other reward, or is a place on the leaderboard more than enough of an incentive to keep playing? Personally though, I think the linked article about on-the-fly resizing[0] was a more interesting "technical" read. [0] http://code.flickr.net/2015/06/25/real-time-resizing-of-flickr-images-…

Love it. I hope they're able to publish the data. If only parts of it are generated using Google Maps data (see the 'create' feature in the screenshots) then they wouldn't because it's derived work. Here is Flickr equivalent http://code.flickr.net/2008/10/30/the-shape-of-alpha/ (downloadable). Foursquare also does similar work. [edit: sorry, I thought Airbnb was analyzing listing data to find the extends of neighborhoods. They don't seem to do that.]

From my experience working with PHP for over 13 years (since PHP 3 was new), I can say your article is mostly true, and I have definitely seen the invisible-image-as-a-poor-mans-cron trick in production a time or two. The only nitpick is that I have really only seen stuff like this on small sites that don't have enough traffic for it to ever matter, so it's generally a temporary issue or a stop-gap measure, not common practice as was implied. That said, there's nothing stopping people from just throwing a task in Redis from their PHP frontend and using node.js or Python, or Ruby to manage the queue. This is what Flickr did (http://code.flickr.net/2012/12/12/highly-available-real-time-notificat… and it seems to work well for them. People just have to keep in mind what each language excels at and know the drawbacks. Articles like yours help, so thanks for the good read.

I wasn't familiar with what 'Flickr tickets' were so I did a little Google-fu and found this excellent explanation: http://code.flickr.com/blog/2010/02/08/ticket-servers-distributed-uniq…

San Francisco, CA - Flickr We recently had our 6 billionth photo uploaded and we (the engineers) just built and rolled-out the new geofences feature. Engineers really get to make a difference at Flickr on a daily basis. http://code.flickr.com/blog/2011/08/30/in-the-privacy-of-our-homes/ (That's my kitchen.) We're hiring backend engineers, designers, and operations. Drop me a line at caudill -at- yahoo-inc.com if you're interested. I'd love to talk to you.

Foursquare went with OpenStreetMap though, and claim "while the new Google Maps API pricing was the reason we initially started looking into other solutions, we ultimately ended up switching because, after all our research and testing, OpenStreetMap and MapBox was simply the best fit for us." http://blog.foursquare.com/2012/02/29/foursquare-is-joining-the-openst… They could be lying to cover a penny-pinching move, but I think they're telling the truth simply because Foursquare doesn't need particularly good geodata. It only needs for locations you visit to be fairly recognizable and locatable, it doesn't need to know that you can't do a u-turn or go that way up a one-way street or the exact location of a point to less than 1m accuracy. The only businesses it cares about it wants in its own dataset, not the map providers. Same story for Apple and its photo display or (going way back now) Yahoo/Flickr using OSM for geolocating photos in Iraq and China, Afghanistan etc. despite Yahoo having their own map team: http://code.flickr.com/blog/2009/02/13/changelog-yahoo-updated-map-til… This process has been going on for a while now, it's the traditional story of the open alternative eating the market from the bottom-upwards and accelerating as it gets wider use and improves in quality in a virtuous circle.

Every linguist will tell you that language is always mutating and any dictionary is a biased snapshot of how language is actually being used. The same is true for geography. Locals always know the quickest routes, the shortcuts, the bike-friendly streets. Top-down approaches like Tele Atlas and Navteq can miss these subtleties. OpenStreetMap is brilliant not only because it's crowd-sourced mapping, but because it allows the crowd to define and create new feature types and place-tags on-the-fly. This means you get a map that shows you where footpaths and dog parks are, but it also means that there's infinite room for connection with other services. Flickr's support for OSM tags is a striking example: http://code.flickr.com/blog/2009/09/28/thats-maybe-a-bit-too-dorky-eve… I should also mention that some of my university colleagues are working on map wiki for cyclists in Minneapolis. http://cyclopath.org It is focused on being a research platform for collaborative map-making.

Flickr did it in 2014, same year as the comic. Unfortunately the service is down and they didn't include a screenshot of it working. https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Everything old is new again: > Flickr is somewhat unique in that it uses a code repository with no branches; everything is checked into head, and head is pushed to production several times a day. This works well for bug fixes that we want to go out immediately, but presents a problem when we’re working on a new feature that takes several months to complete. How do we solve that problem? With flags and flippers! https://code.flickr.net/2009/12/02/flipping-out/

Iirc, Flickr had implement bird detection within a few months of this xkcd coming out? EDIT: A month, https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/ It's so weird them explaining 'Deep Networks'. Language on AI has definitely changed in the past ten years. Also, hilariously, the page they created to demonstrate this (http://parkorbird.flickr.com/) no longer works. Oh, how time flies. explain XKCD page for good measure: https://www.explainxkcd.com/wiki/index.php/1425:_Tasks

edit: found the flickr link - https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Small correction - it wasn't until 2014 that it was widely available in libraries. AlexNet was September 2012. FlickR did their XKCD1425 bird detector in Oct 2014: https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Randall Monroe himself didn't know how hard (easy) the task was. The Flickr team released a month after the comic this (sadly offline now): https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/ Today a hobby programmer could do it by themself.

For sometime there were also "machine tags", basically a triple tag invented (I think) at Flickr[0]. It was an interesting concept, you could automate relationships between different contexts, for example between Flickr and Last.fm[1]. I used it for a while, then I always wondered why nothing similar has ever emerged, maybe because after the first wave of "social sharing" excitement of web 2.0, every walled garden has basically double locked their gates. And this is maybe what happened to tagging in general. [0]: http://tagaholic.me/2009/03/26/what-are-machine-tags.html [1]: https://code.flickr.net/2008/08/28/machine-tags-lastfm-and-rocknroll/

I have been using Flickr to archive my photos since 2013 when they announced a 1TB limit for free users, and have been paying them $50/year since 2016 when they limited availability to Pro users for their first-party auto-upload tool ( http://flickr.com/uploadr ). It did seem like the 1TB limit was too good to last when it was announced 5 years ago. Still, it's a relatively inexpensive extra archive for my photos (400GB) that supplements other backups. The archive is sorta searchable and kinda good for sharing with family and friends. I haven't really used their social or community features (the Explore experience, the magic donkey, and the pandas are all alien to me -- https://www.flickr.com/photos/thomashawk/361974994 , http://code.flickr.net/2009/03/03/panda-tuesday-the-history-of-the-pan… ).

Arguably, it's still relevant. It took about 5 years for suitable image recognition apis to come out. Although Flickr tried to get it done in about a month: http://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

> significant overhead cost for anyone who needs to handle images from unconstrained sources Flickr is doing just that, and they’ve been using GPUs for more than 2 years already: http://code.flickr.net/2015/06/25/real-time-resizing-of-flickr-images-… > It'll happen but this kind of thing always takes longer than it seems like it should. I think the main reason for that is lazy software developers reluctant to learn new stuff.

Essentially, I was going down this route if anyone's curious.... http://code.flickr.net/2012/12/12/highly-available-real-time-notificat…

it's a feature flip/toggle service http://martinfowler.com/bliki/FeatureToggle.html https://code.flickr.net/2009/12/02/flipping-out/

Yeah, Randy was a bit miscalculating the estimate there. Within a month of that comic, flickr made http://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Flickr's response to the xkcd: https://code.flickr.net/2014/10/20/introducing-flickr-park-or-bird/

Another (old) example of how Flickr does (or quite possibly "did") this http://code.flickr.net/2010/02/08/ticket-servers-distributed-unique-pr…

None. Continuous integration and feature flags FTW: http://code.flickr.net/2009/12/02/flipping-out/

No, no, no, no, no! This is a terrible idea, and will end up causing you a massive headache in the long-run. If I see commented-out code like this, I will delete it because that's what version control is for. If you really need to turn code on/off in different environments, you should be using feature flags: http://code.flickr.net/2009/12/02/flipping-out/ and at the very least you should be detecting the environment: if (ENV == 'dev') testStuff(); else productionStuff(); This is much more meaningful to other developers (and to yourself in the future).

I think I know the thing you mean, but I can't find it either. I did find 'flippers' from Flickr[1] which sounds like a very similar "keep it all in master and turn stuff on and off" methodology. [1] http://code.flickr.com/blog/2009/12/02/flipping-out/

Flickr has had 5 or 6 major updates this year, including a rebuilt uploader, redesigned photo page, a new justified view, social album browsing, integration of a new photo editor, and more. So, there is a lot of work being done at Flickr. See announcements @ http://code.flickr.com/blog/ If there's anything specific you'd like to see, leave a reply.