Shopify Engineering

Shopify | Developer | Toronto, Ottawa, Montreal, Waterloo | FULL-TIME ONSITE VISA| http://shopify.com Shopify is a platform that allows entrepreneurs to easily setup an online store. Our products help merchants who are just starting as well as established brands that need a solution that can scale with their traffic. We're always working on products that make it easier for entrepreneurs to reach their audience and help them make data driven decisions. Shopify is built in Ruby on Rails running on a stack composed of Docker, Golang, Mysql and Redis. Our data infrastructure uses Kafka, HDFS, Zookeeper and we use PySpark and Sklearn for our data modeling and machine learning tasks. If you're interested in building tools that empower Entrepreneurs come take a look at who we are and what we're doing - blog: https://engineering.shopify.com/ - github: https://github.com/Shopify - job postings: https://jobs.lever.co/shopify?lever-via=XBuWsYM_Q2 Regardless of where you're applying this year I'd say make sure you focus your application on specific roles, highlight your work and personal projects (over academics) and don't treat interview results as a reflection of personal worth. Good luck with your job hunt !.

I've found Shopify's blog interesting (and I don't happen to use Shopify or have any affiliation with them): https://shopify.engineering

Very interesting blog post! My best guess as to how shopify is able to achieve such large increases in speed is they have a lot more actual ruby code than most codebases (https://shopify.engineering/ruby-yjit-is-production-ready).

It really depends what you design your system to handle. A sudden burst of traffic that can be spread for minutes is fine (ex: Elasticsearch indexing requests can usually be delayed through background jobs). > Basically those things only sort of work if you have few messages and are happy with millisecond latency. Not really... queues are great to defer processing or have longer running job than your HTTP and TCP timeouts will allow. Building large data export won't happen within a single HTTP request. ex: https://shopify.engineering/high-availability-background-jobs

I second the recommendation for zero-to-nix, it's a great resource. I really like what DetSys is doing for the Nix community! In addition to that, check out this Shopify video series: - https://www.youtube.com/playlist?list=PLRGI9KQ3_HP_OFRG6R-p4iFgMSK1t5B… - https://shopify.engineering/what-is-nix I share a bunch of configuration between my M1 Macbook and x86 desktop in my office. I use home-manager with config that looks like nixos/*, macos/*, and common/*. The first two just import the latter. Here's a link if it's helpful: - https://github.com/dustinlyons/nixos-config

Just search for TCP handshake on Youtube. Then install a program called Wireshark. Search Youtube on how to use it to inspect traffic. Then read this excellent article on how web frameworks actually work: https://shopify.engineering/building-web-app-ruby-rails Unless you're looking to build your own web framework or something that's all you need to know for backend development because everything is nicely encapsulated these days. If you want to build frameworks and stuff then learn C and grab a book called TCP/IP Illustrated.

Facebook definitely does it. Thats where we first learned about this way of working. Some bigger tech companies have actually written about it (see links below) but a list of ones that we've seen with this kind of infra include LinkedIn, Shopify, Dropbox, Stripe, Slack, DataDog, and Palantir - to name a few. Eng blog links: LinkedIn: https://engineering.linkedin.com/blog/2021/building-in-the-cloud-with-… Shopify: https://shopify.engineering/modelling-developer-infrastructure-teams Slack: https://slack.engineering/development-environments-at-slack/ Doximity: https://technology.doximity.com/articles/github-codespaces-is-great-bu…

https://shopify.engineering/ https://developer.squareup.com/blog/ https://netflixtechblog.com/ https://githubengineering.com/

Shopify (https://www.shopify.com) | Frontend / Backend | UK and Europe Come and help us make commerce better for everyone. We're expanding into EMEA and I'm hiring out the team here. I'd love to work with you. We're fully remote (https://www.shopify.com/careers/work-anywhere) and have a fantastic engineering culture (https://shopify.engineering/) EMEA engineering roles I'm hiring for: https://bit.ly/3l0Sfut If you'd like to talk to me directly, then you can email me at james dot stanier @shopify.com, ping me on Twitter @jstanier, or find me on LinkedIn: https://www.linkedin.com/in/jstanier/

There are probably a lot of benefits to be gained from Rails on Wasm. For example, Shopify recently posted an article about how they use Wasm outside of the browser to execute untrusted code (user scripts) [0]. It is also worth mentioning that although Artichoke supports WebAssembly as a build target, compiling to Wasm is not the only way to execute an Artichoke program. 0: https://shopify.engineering/shopify-webassembly

Hey Leafboi - I recommend reading the first post in the series for some background https://engineering.shopify.com/blogs/engineering/deconstructing-monol… We don't use "hardware" or "VMs" to facilitate modularity.

Hopping on the topic of addresses and not completely related to your point - but if anyone's having challenges with handling address fields I'd highly recommend checking out Shopify's address package: https://github.com/Shopify/quilt/tree/master/packages/address And corresponding blog post: https://engineering.shopify.com/blogs/engineering/handling-addresses-f…

It totally depends on task at hand. If you have a db/forms heavy project than Rails/Django gives you advantage, if your project is more driven by interactions and subtle UX then SPA lets you focus on that. Like, imagine how hard it would be to make Shopify as SPA (https://engineering.shopify.com/17489056-rebuilding-the-shopify-admin-… On the other hand, Trello would never feel so fluid if it was driven by server rendering (notice how weird Github projects comparing to Trello). Personally, I've been doing SPA for years and think it's the right way for reach UI. But recently I had to build a form-heavy app and decided to go with classical server-side setup + some jQuery on client. Old school but very productive. EDIT: added link to shopify blog.

> But are they more productive, get shit that we care about done [...] I think there's a balance to be struck between "getting shit done" and "write something that will be maintainable in the future". It might be true that (some?) Haskellers overdo it, but I've definitely seen enough of the "let's just get shit done" people who create messes that are completely unmaintainable and full of bugs after just a year or two. And IMHO, the software industry is in a really sorry state, considering how bug-riddled most software is, how most software projects fail or are considerably above budget, etc. [1] I've seen quite a number of failed projects in the companies I've been so far, a number of them involving quite a lot of resources. I've seen companies trying to rewrite hopeless messes and failing at it, grinding everything to a halt, because the code base is full of implicit assumptions that nobody bothered to properly specify anywhere. I've read about Shopify spending a whole year to upgrade to a newer version of Ruby on Rails.[2] Probably the industry doesn't yet know how best to find this balance but I don't think that the Haskellers are solely to blame for this. [1]: https://danluu.com/everything-is-broken/ [2]: https://shopify.engineering/upgrading-shopify-to-rails-5-0

Appreciate the feedback and recommendation! You're right that explicit judgments can get stale - fortunately for our document collection the information architecture and article structures themselves are slow-changing (the answers themselves might change, but the document that answers the question probably won't for some time). We also primarily use explicit judgments to label head queries/common topics, and may augment our datasets with fresh data from time to time. The team is currently exploring augmenting these human-made datasets with automatic judgments using click models. For realtime feedback, we've implemented (on another search product at Shopify, not the Help Center) a "near"-time feedback loop using implicit judgments to alter search results. Perhaps I'll write a post about that one soon :) . My colleague Doug talks a bit about the new systems we're building in this blog post - https://shopify.engineering/apache-beam-for-search-getting-started-by-…

Hi @simonhamp, founder/CEO of Akita here. :) It sounds like you have good internal discipline about what could potentially break your API and that's great. The point you bring up, that surely the code can't be the only documentation of an implicit contract, is exactly what we've been observing, in that the code is often the only documentation. The example we gave (service/service dependency) was an extreme case of this, though we have seen this. More subtle examples would be changing an error code, or changing from one specific type of string to another (for instance, two datetime formats). Propagating these changes to dependencies, especially in larger systems, can cause some real headaches. Would love to talk more to understand how you've gotten around some of these with good process! Shopify has this great blog post about their solution for change management in their external APIs that also makes the point that change impact analysis is not easy: https://engineering.shopify.com/blogs/engineering/shopify-manages-api-… Also, quick point: we don't proxy for precisely the reason you brought up. Akita works without sending any user data back to our servers, just the metadata.

Shopify | Developers and Designers! | Toronto, Ottawa, Montreal, Waterloo | FULL-TIME ONSITE VISA| http://shopify.com Shopify is a platform that allows entrepreneurs to easily setup an online store. Our products help merchants who are just starting as well as established brands that need a solution that can scale with their traffic. For devs: Shopify is built in Ruby on Rails running on a stack composed of Docker, Golang, Mysql and Redis. Our data infrastructure uses Kafka, HDFS, Zookeeper and we use PySpark and Sklearn for our data modeling and machine learning tasks. For design: Our design teams are always growing, we’re improving the UX of commerce for merchants all over the world. We built Polaris, our design system which helps us build consistent experiences for all merchants. If you're interested in building tools that empower Entrepreneurs come take a look at who we are and what we're doing: - blogs: https://engineering.shopify.com/, https://ux.shopify.com/ - job postings: https://jobs.lever.co/shopify?lever-via=1_PEVU3ujn Our interview process isn’t super high pressure, and I think we’ve done a good job so far extending culture and finding good hires. I’d suggest applying even if you aren’t a perfect fit - I was in an weird university situation but was still taken seriously. Cool projects are always a good talking point. Good luck :)

castform founder here. while it's "rare for a specialized model to beat a strong general model" today, i think the tech/knowhow on how to do so is getting there. we see some early signs of this in industry e.g. https://shopify.engineering/sidekicks-continual-learning-loop https://thinkingmachines.ai/news/learning-to-replicate-expert-judgment… https://castform.com/blog/elsa/ at the end of the day, models are only as good as the data they're trained on. and if one has access to proprietary data, it should yield specialized models that do better than larger general models

This is essentially what Shopify did with River (https://shopify.engineering/under-the-river), and we built out something similar at our company. I'm sure the labs are working on this sort of thing - Managed Agents are probably the closest?

What do you mean? The whole point of Ruby on Rails is the rails way? Also the problems you are describing are not new and the community settled on adding some sort of service layer https://shopify.engineering/shopify-monolith http://sporto.github.com/blog/2012/11/15/a-pattern-for-service-objects…

I mean, this is "fundamentally" wrong. Shopify for example continues to use React Native [0]. [0] https://shopify.engineering/five-years-of-react-native-at-shopify

This article and video might be good for you, but they are more about a conceptual overview of what the fundamental parts of Nix are and how they interact. https://shopify.engineering/shipit-presents-how-shopify-uses-nix https://www.youtube.com/watch?v=6iVXaqUfHi4 Flakes are a dependency-pinning and hermetic evaluation feature baked into the Nix tool itself. Overlays are an idiom that uses lazy evaluation to build up larger attribute sets in a modular and composable way (e.g., package collections for a particular programming language).

You're spot on! We (Shopify) are using it now: https://shopify.engineering/mcp-ui-breaking-the-text-wall

Cool idea. I thought I’d try to make a Kagi Lens to accomplish the same thing: https://kagi.com/lenses/LdYine8hZtYmrt8yTMngOUtvTM9rmkRy Kagi Lenses can be defined in many ways, one of which is specifying URLs to search. Unfortunately you can only provide 10 URLs per lens. Here are the ones I chose: https://stripe.com/blog/engineering, https://engineering.fb.com/, https://www.uber.com/en-US/blog/engineering/, https://netflixtechblog.com/, https://research.google/blog/, https://technology.riotgames.com/, https://incident.io/blog, https://www.anthropic.com/engineering, https://openai.com/news/, https://shopify.engineering/

Funding? Seems like the development was funded by Shopify and they got a ~20% performance improvement. https://shopify.engineering/ruby-yjit-is-production-ready A similar experience in the Python community is that Microsoft funded "Faster CPython" and they made Python 20-40% faster.

A recent improvement in import map support in the browser https://shopify.engineering/resilient-import-maps

Regarding how a bunch of the pieces go together, I found this blog post from Shopify quite useful: https://shopify.engineering/what-is-nix

Shopify made the new Ruby JIT compiler. [0] They're on the Rails Foundation, as is 1Password, among others. Stripe is still in on Ruby too; they're behind the Sorbet gradual type system, and Ruby is a first-class language for working with their API. I always hear the stereotype of companies starting on Rails and migrating later, and I think it sticks around because it makes some level of intuitive sense, but it doesn't actually appear to happen all that often. I guess successful companies don't see the need to rewrite a successful codebase when they can just put good engineers to work on optimising it instead. [0] https://shopify.engineering/ruby-yjit-is-production-ready

You'd hope they benchmarked the old native iOS app and the RN app. Since the blog post doesn't mention previous native-only perf, I'd assume they didn't compare or the RN version isn't close to native-only perf (leaning heavily towards the second reason). Looking at a previous blog post, the first hunch seems to be correct - the second may also be true. From 2024 March, https://shopify.engineering/improving-shopify-app-s-performance talks about how their RN-ified app was loading screens in 1400ms (P75) and the steps they took to reduce that to 500ms. I hope they benchmark their load-screen time with every release/CD to stay on top of any regressions, otherwise, there'll be more mad scrambles when the perf debt piles up too high.

500ms is the 75th percentile speed, so 75% of users are having load times faster than that. For context, Google's synthetic p75 loads emulate a crappy old Android phone on a bad network. A linked post[0] says their p75 was 1400ms before 2023, yowza. [0] https://shopify.engineering/improving-shopify-app-s-performance