Spotify Labs

There are a bunch of great references for career ladders, broadly speaking, that discuss the rollout process and what each level means for that company. The best not only say something about the levels, but about how the levels are interpreted in line with that company's culture, whatever it means. It's very hard to get to levels beyond senior as a pure generalist: you're typically going to have to have worked on projects with significant, company-wide impact, and that usually means (a) doing something unique within a domain and (b) working with many others, in different types of roles, in your company, in order to have that impact. Two good intro readings on career ladders: https://labs.spotify.com/2016/02/08/technical-career-path/ http://dresscode.renttherunway.com/blog/ladder

This is a great idea. If you couple this practice with storing user data encryption keys with additional layers of security, you’ll decrease your susceptibility of someone being able to extract all data if they get access to your kafka. Spotify talks about this here: http://labs.spotify.com/2018/09/18/scalable-user-privacy/

Worth mentioning Spotify's account hijacking problem when using unicode https://labs.spotify.com/2013/06/18/creative-usernames/

Valueless content finds you. Worthwhile content you really have to go looking for, or have a good social network funneling stuff to you. My own answer is mostly HN, Twitter (including a couple of people/bots who are great link sources), and a mailing list of some old friends for this purpose. The latter is very good and low volume; eg latest post was from 18 July on org structure for software projects and lists https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1… https://www.infoq.com/news/2016/10/no-spotify-model# https://www.slideshare.net/AliKheyrollahi/microservice-architecture-at…

Spotify wrote a brilliant blog post on this. Highly recommend reading. https://labs.spotify.com/2014/02/28/how-to-shuffle-songs/

From time to time, I'v enjoyed Spotify's blog: https://labs.spotify.com/

I work at Spotify on ML. I spent quite sometime working on our production vector model pipelines. We have three components: 1. Training data: Scalding pipelines (could be any large scale production framework) to preprocess training data and reduce it to a few hundred gigs. 2. Run latent vector models (Matrix factorization, Embedding models etc) to generate item vectors. This stage is designed to be non specific to framework. We use typed avro objects to communicate with the previous stage. 3. Production deployment of models. Essentially glue code that ensures that models are versioned, atomic (user, music vectors are all from the same batch) and deployed into music recommendation and realtime frameworks. Read here[1] more. [1] https://labs.spotify.com/2016/08/07/commodity-music-ml-services/

Shipping python projects isn't easy. Even if you have a prefect package on pypi, that specifies its dependencies, it may conflict with other tools using other versions of those libraries. Compiling python does not feel right. Using native packages is better[1], but you have to plan it out. You can package virtualenvs into native packages[2] - and this is probably the most solid way, but you'll need packaging setups for various package managers. In short, "written in Go" for me means with a high probability, just run "go get x" - or download a binary. Since we are at it, I'd love to hear about better shipping strategies for Python projects. [1] https://hynek.me/talks/python-deployments/ [2] https://labs.spotify.com/2013/10/10/packaging-in-your-packaging-dh-vir…

Reminds me of a issue Spotify had too with unicodes and usernames. https://labs.spotify.com/2013/06/18/creative-usernames/ I had someone tell me that programming isn't real work before, and this is yet another example of all the small little details going into building things that most people don't really think about day to day. I haven't had to work with login code in a while, but might at some point. I know some systems only allow alphanumeric usernames, but sounds like can't force people to have alphanumeric emails... Well I guess you could but might upset someone. I know there's normalizing functions though like NFD, NFC, NFKD or NFKC that might work for usernames but not sure what's really recommended. Then also brute forcing attempts to try to mitigate attacks and other considerations to make also when building out an account system. Then if your company is large enough to provide phone support, not sure how you'd tell the support person which specific emoji you used in your username.

NYC Spotify employee here. Spotify has a very honest, open, and autonomous culture. Spotify is comprised of many Missions which are defined by a business purpose like ads, user engagement, or analytics. Within a Mission you will have one or more Tribes. Each Tribe is loosely associated with the Mission goal and addresses a portion of the overall Mission. A Squad is a delivery team/product team. Squads are accountable for building great products. They are given autonomy to make the right decisions for the direction of their product. Squads function as "mini-startups", if you will. A grouping of many Squads is called a Tribe. A Mission will have one or more Tribes. It's a tad confusing and I'm sure I've gotten some of definitions wrong, but you get the general gist. As for the people culture, Spotify values the growth of employees. We focus heavily on personal growth over product delivery. We believe if we build good people they will build good products. I was doubtful before joining Spotify that they would fulfill everything I was told (about the culture) during the interview process but everything has held true. For better descriptions of Spotify check out the engineering culture video: https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1… I would encourage you to apply. Spotify is a great place to work.

This is correct link https://labs.spotify.com/2020/02/18/wrapping-up-the-decade-a-data-stor…

This may be a more appropriate source, from the source: https://labs.spotify.com/2019/11/12/spotifys-event-delivery-life-in-th…

IMO, an infinite queue does not really apply to VLC. If all songs are correctly tagged (which cannot be assumed in VLC), what Spotify does is interesting: https://labs.spotify.com/2014/02/28/how-to-shuffle-songs/

Spotify has a nice blog post about shuffling music. Not the exact same thing, but interesting nonetheless. https://labs.spotify.com/2014/02/28/how-to-shuffle-songs/

There's a gamut of reasons, and it benefits all of managers, ICs, and the company. I like to point to Spotify's prose on building their engineering career ladder as a good example of why ladders can be important. 0. https://labs.spotify.com/2016/02/08/technical-career-path/ 1. https://labs.spotify.com/2016/02/22/things-we-learned-creating-technol…

Missed one! Spotify just released their own dataviz library yesterday https://labs.spotify.com/2018/11/15/introducing-chartify-easier-chart-…

Reminds me of the Spotify unicode username issue[1]. Where a function assumed to be idempotent (think tolower(username)) actually wasn't with certain unicode inputs. Allowing account takeovers. [1] https://labs.spotify.com/2013/06/18/creative-usernames/

Also using similar unicode characters in your username for nefarious spoofing attacks or worse -- https://labs.spotify.com/2013/06/18/creative-usernames/ This is called a Homoglyph attack. If you accept unicode for strings that should be "unique" (eg username), there are various normalization schemes that basically convert equivalent-ish looking characters into a consistent hash. I have no doubt spam filters use this.

Creator here. A little background on why I made this and how it could be used to add some whimsy to data visualizations: https://labs.spotify.com/2018/03/02/introducing-coordinator-a-new-open…

I don't know your flow, but how about using a queue system to prevent the bursts of data? We're internally using pubsub (and nats) sending billions of messages every day. We don't use the push mode as Spotify [0] but pull from the queue instead which allows us to run on our pace. We do write into bigquery (and citus) and each components is doing it on their own pace based on what they're capable of process at the moment. [0] https://labs.spotify.com/2017/11/20/autoscaling-pub-sub-consumers/

Creative usernames and Spotify account hijacking Posted on June 18, 2013 https://labs.spotify.com/2013/06/18/creative-usernames/

Agile principles generalize to all software development. The specific application of those principles at one company won’t. That’s the core difference people miss. Of course then there is the difference between “software development” and “one individual programmer”. Not all programmers will gel with Agile stuff as well as the next one. That’s why it’s not supposed to be rigid (it often is when done poorly). But yes, an understanding of the principles is required to translate them into practices for a team. Rigid adherence to Scrum or XP is missing the point. For a good look at a way one company took the principles of Agile and applied them in a very effective but utterly idiosyncratic way... look at Spotify. Pt 1: https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1… Pt 2: https://labs.spotify.com/2014/09/20/spotify-engineering-culture-part-2…

Here's a link that actually worked for me: https://labs.spotify.com/2014/02/28/how-to-shuffle-songs/ Also, the title should include [2014]

[disclaimer: I work at Google] https://cloud.google.com/customers/ Spotify, SnapChat, eBay are the examples I usually give when asked that question. I'm not sure what numbers I am allowed to provide for any of them, but there's some public information available that gives you a sense of the scale involved: https://labs.spotify.com/2016/03/03/spotifys-event-delivery-the-road-t… https://www.itnews.com.au/news/do-not-fear-the-cloud-ebay-migrates-113…

Spotify sounds very interesting to me. But I don't work there, so cannot really tell if it really works as they describe. https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1… https://labs.spotify.com/2014/09/20/spotify-engineering-culture-part-2…

Also note Spotify published their own stuff also on Arista hardware https://labs.spotify.com/2016/01/26/sdn-internet-router-part-1/ Podcast where David Barroso talks about it: http://blog.ipspace.net/2015/01/sdn-router-spotify-on-software-gone-wi… Arista blog post: https://eos.arista.com/spotifys-sdn-internet-router/ Disclaimer I work at Arista

You can listen and read Spotify engineers talking about their experiences with Google, including running 2,000,000 qps load tests without ever telling anyone at Google. If you read some of their blogs, you can see how brutally honest they are about the pros and cons of our services: https://labs.spotify.com/2016/03/10/spotifys-event-delivery-the-road-t… https://www.youtube.com/watch?v=LTVFg6YOjWo&index=7&list=PLIiv… https://www.youtube.com/watch?v=xT6tQAIywFQ&index=1&list=PLIiv… Finally, does this sound like an engineer who got paid to speak? https://twitter.com/mpjme/status/702167231623516160

It's a ballpark feel for the org. An admitted problem with my theory is that I've only seen behemoths (massive, nasty publicly traded beasties) and start-up to mid/small orgs with only 1-200 ppl having access to source control and/or servers. If an org can grow beyond the tribal size & still maintain trust & agility - please share! I've got a good theory about how Spotify operates (see https://labs.spotify.com/2014/03/27/spotify-engineering-culture-part-1… but I don't know if they've effectively crossed or exceeded this scope of personal trust & relationships.