Chris Richardson

Forgot to say, you should also check out Chris Richardon's excellent website dedicated to microservices: https://microservices.io/

This is my gold reference: https://microservices.io/ Most issues I have seen around microservices circle around bad design decisions or poor implementations. Both are also major issues for monoliths, but since those issues do not surface early on, it is easier to start developing a monolith and take on the technical debt to be dealt with later on. Microservices architecture takes time and effort, but pays huge dividends in the long run.

O’Reilly Building Event-Driven Microservices is a solid book to continue. I would not recommend Architecture Patterns with Python. Some patterns there are questionable and introduce more complexity. Also, they talk about using Kafka and Service Bus in the book, but never mention an important issue you may face in real implementation: consistency in updating both a message bus and a database that usually is addressed with Outbox pattern https://microservices.io/patterns/data/transactional-outbox.html

It's industry best practice -- not a joke. What's more, the recommendation is to have a separate database per service -- one table, one service, one database. Fuethermore, when a join is desired, the best practice is to implement not just a service that joins the data but maintains it in a table/materialized view of its own, along with a message server such as Kafka. The services responsible for the tables to be joined (customers and orders, for instance) put events on this message bus which the joining service subscribes to in order to know when to update its view. See: https://microservices.io/patterns/data/cqrs.html And don't get me started on how you do transactions that touch multiple tables in this setup: https://microservices.io/patterns/data/saga.html I have seen these deployed and advocated as the modern way to write business applications in the wild.

If your application is stateless following your definition then it's trivial to break the monolith down to microservices that focus on a bounded contexts while sharing a common db. This is a very well known microservices pattern. https://microservices.io/patterns/data/shared-database.html

One place to start is Chris Richardson's http://microservices.io

To all the folks saying “durable execution frameworks solve this”—you’re right, but a lot of what’s described in the article isn’t quite the same as durable execution a la temporal. The approach described (transactional outboxes for side effectful operations, and care taken to be idempotent or resumable where possible, and to gracefully degrade, slow down, or rate limit where you can) achieves some of the same properties as a given durable execution framework, its true, but you don’t necessarily need to rewrite your code to be fully event sourced or use a framework to get a lot of those benefits, as the article demonstrates. Transactional outboxes specifically are one of my favorite patterns: they’re not too hard to add and don’t require changing many core invariants of your system. If you already use some sort of message bus or queue, making publishes to it transactional under a given RDBMS is often as simple as adding some client side code and making sure that logical message deduplication and is present where appropriate: https://microservices.io/patterns/data/transactional-outbox.html If you use a separate message broker (Kafka, SQS, RabbitMQ) with this pattern, you’ll also need a sweeper cron job to re-dispatch failed publishes from the outbox table(s) as well. Bonus points if this can be implemented on top of existing trigger-based audit table functionality.

> Once the code for all destinations lived in a single repo, they could be merged into a single service. With every destination living in one service, our developer productivity substantially improved. We no longer had to deploy 140+ services for a change to one of the shared libraries. One engineer can deploy the service in a matter of minutes. This is the problem with the undefined nature of the term `microservices`, In my experience if you can't develop in a way that allows you to deploy all services independently and without coordination between services, it may not be a good fit for your orgs needs. In the parent SOA(v2), what they described is a well known anti-pattern: [0] Application Silos to SOA Silos * Doing SOA right is not just about technology. It also requires optimal cross-team communications. Web Service Sprawl * Create services only where and when they are needed. Target areas of greatest ROI, and avoid the service sprawl headache. If you cannot, due to technical or political reasons, retain the ability to independently deploy a service, no matter if you choose to actually independently deploy, you will not gain most of the advantages that were the original selling point of microservices, which had to do more with organizational scaling than technical conserns. There are other reasons to consider the pattern, especially due to the tooling available, but it is simply not a silver bullet. And yes, I get that not everyone is going…

Go out and buy/find/read the following books: (1) Patterns of Enterprise Software Architecture https://www.oreilly.com/library/view/patterns-of-enterprise/0321127420… WHY: What kind of things do we need to build enterprise software applications, and where should we draw lines to separate concerns for these subsystems? (2) Gang of Four Objected oriented design patterns https://www.digitalocean.com/community/tutorials/gangs-of-four-gof-des… WHY: what are the recurring themes in doing good OOP as described by the guys who have been doing it since the beginning? (3) Microservice Design Patterns https://microservices.io/patterns/microservices.html WHY: What are the recurring patterns, themes, and challenges in building good micro service architectures? (4) Uncle Bob’s Clean Code https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship WHY: Write code that somebody else could pick up, read, and not be completely baffled by. In all likelihood, the next person to read your code will be you. Here’s how to do it. (5) SOLID Design patterns WHY: Essentially how to do good OOP part deux. These are basically, in my opinion, the equivalent of University Physics or Eisberg/Resnick QM textbooks. They’re seminal texts with a lot of valuable insights into patterns, jargon that other seniors throw around, etc. Also depending on your learning style I’ve seen this material covered in a billion different shapes: YouTube pseudo-lectures, GitHub repos, blog…

This seems appropriate... https://microservices.io/

Gateway APIs: http://microservices.io/patterns/apigateway.html Assuming you've read that then I'd add the following. If you don't have something approaching a gateway, you're probably doing a big disservice to your users (assuming clients fetch data via API, rather than using a traditional render-on-the-server framework like Django or Rails). The value proposition of GraphQL is based on the assumption you're already doing things right by users, and its possible to fulfill the data requirements for a single page or app screen in a single API call. What this probably means is: * You have a custom endpoint per screen * That endpoint to either a) be versioned or b) support as many historical versions of your app as are in production * You might have a gateway per client (this is the full backend-for-frontend pattern) * Every time you add functionality, or change existing functionality, you're adding to what quickly becomes a huge set of endpoints What GraphQL promises (and delivers on) is the ability to get all the benefits of the backend-for-frontend pattern, without anyone ever having to write an endpoint specifically for a give client use case. The clients convert their data requirements into a query, and the GraphQL server returns the exact data you need, no more, no less. It requires some discipline when it comes to evolving the schema over time, but it works really really well. And when implemented intelligently has comparable performance to hand-crafted endpoints…

You can get most out of it by using it as a "sidecar" next to your native Rust application. That way you get access to the full power of Rust by triggering "native actions" from flawless through http. And you can still use flawless for code that you think is more important and you want to always complete and never silently drop/fail mid-execution. This is also very useful if you have a micro-service architecture and you need to call multiple micro-services to end up in a consistent state. It's a way to reliably implement the saga pattern: https://microservices.io/patterns/data/saga.html

A workflow manager implements a Choreography based saga pattern https://microservices.io/patterns/data/saga.html

https://microservices.io/patterns/data/transactional-outbox.html It allows you to wrap it all in a transaction. If you separate a database update and insertion into a queue, whichever happens second may fail, while the first succeeds.

Something to think about: https://microservices.io/patterns/data/cqrs.html

Nice problem observation. One solution is the outbox pattern: https://microservices.io/patterns/data/transactional-outbox.html

> falsely concludes i disagree here. instance b did the right thing given the information it has. instance a should realize it no longer owns the lock and stop proceeding. but in reality it also signifies concurrency based limitations in the api itself (no ability to perform a do and then commit call). https://microservices.io/patterns/data/saga.html

One pattern is having a Workflow that runs as long as the lifetime of a domain object and holds a conceptual lock on that object—it receives requests to modify the object, and makes sure to only perform one operation at a time. (like the state pattern on a particular agent) Also related: Signals are events that you can send to Workflows and between Workflows, and they’re always delivered in the order they’re received. More generally, for a handy reference of Distributed Systems patterns, check out https://microservices.io/patterns/data/saga.html (though I personally find his diagrams a bit...overwhelming) and the MSN writeups: https://learn.microsoft.com/en-us/azure/architecture/patterns/

It's got a better name called a transactional outbox https://microservices.io/patterns/data/transactional-outbox.html

Yes indeed. Attaching below a couple of references from my exploration: Transactional outbox pattern: - https://microservices.io/patterns/data/transactional-outbox.html - https://softwaremill.com/microservices-101 - https://brandur.org/job-drain A pattern similar to the one I mention using cache: - https://masstransit-project.com/articles/outbox.html

This might help: https://microservices.io/patterns/data/database-per-service.html Varies by actual DB but: with this architecture, if the DBs are NoSQL then you didn't lose as much by splitting up the databases, but if they were SQL then among other things you lose transactions across those DBs (or now need costly distributed transactions), effectively making the collective SQL system behave more like a NoSQL DB.

it's a chicken and egg problem you can either send a kafka message but potentially not commit the db transaction (i.e. an event is published for which the action did not actually occur) or commit the db transaction and potentially not send the kafka message it sounds like they implemented something like the Transactional Outbox pattern https://microservices.io/patterns/data/transactional-outbox.html i.e. you use the db transaction to also commit a record of your intent to send a kafka message - you can then move the actual event sending to a separate process and implement at-least-once semantics This is the job queueing system they described in the article

We are in the middle of the first production-scale deployment of a greenfield microservice-based application now and there are certainly a lot of pain points. On the flip side, I've been a part of about half a dozen extremely successful strangler migrations[0], some of which were downright easy even with a complicated business domain. I often wonder if we would have been better off deploying a monolith in half the time and immediately starting a strangler migration once the rate of business changes slows down. I've become more and more convinced over the past decade that Monolith First + Strangler Migration is most stable, safest way to deliver mid-sized software projects. [0] https://microservices.io/patterns/refactoring/strangler-application.ht…

It is a microservice chassis https://microservices.io/patterns/microservice-chassis.html At scale you have lots of teams writing their own boiler plate and getting it right or wrong. This solves that at the application level. Steel toe is another option by pivotal. You can see an example of how cross cutting concerns can be solved in this mindmap https://www.alanmbarr.com/blog/images/cross_cutting_concerns.jpg

Think he means event sourcing in a distributed transaction context, not a front-end context: https://microservices.io/patterns/data/event-sourcing.html

In all fairness, I have not read this yet. But my initial thought process when I see "a protocol for distributed transactions across microservices" involves a dose of skepticism. If you need distributed transactions across microservices, why not just provide an XA compliant interface, and then plug in to an existing transaction manager? And, probably more to the point, while EBay specifically might need "real" distributed transactions, I suspect that most organizations rarely - if ever - need distributed transactions across microservices, and should instead use the Saga Pattern[1]. [1]: https://microservices.io/patterns/data/saga.html

You could look into functional reactive programming as in rxjs/rxjava etc (https://medium.com/netflix-techblog/reactive-programming-in-the-netfli… or command query responsibility segregation (https://microservices.io/patterns/data/cqrs.html).

I would suggest starting with the short introductory articles written by Chris Richardson: http://microservices.io/articles/index.html A lot of useful articles can be found inside archives section of the Microservices Weekly newsletter: http://microservicesweekly.com/archives

Start here https://martinfowler.com/articles/microservices.html Then look here for patterns http://microservices.io/patterns/microservices.html This guide is great https://www.nginx.com/blog/introduction-to-microservices/ Finally plugging my own tooling https://micro.mu/ Join http://slack.micro.mu if you want to discuss microservices