Shared database in microservices is a problem, yep
This post is in reply to this one: Is a Shared Database in Microservices Actually an Anti-pattern?
The author does a great job outlining the actual problem. Given two services that need to share some data, how do you actually manage that in a microservice architecture? The author uses the Users and Orders example, which is great, because it is pretty simple and require very little domain knowledge.
The first question to ask is: Why?
Why use microservices? Wikipedia says:
The benefit of decomposing an application into different smaller services is that it improves modularity. This makes the application easier to understand, develop, test, and become more resilient to architecture erosion.
I always like an application that is easier to understand, develop and test. Being resilient to architecture erosion is a nice bonus.
The problem is that this kind of architecture isn’t free. A system that is composed of microservices is a system that need to communicate between these services, and that is usually where most of the complexity in such a system reside.
In the Orders service, if we need to access some details about the User, how do we do that?
We can directly call to the Users service, but that creates a strong dependency between the services. If Users is down, then Orders is down. That sort of defeats the purpose of the architecture. It also means that we don’t actually have separate services, we just have exchange the call assembly instruction with RPC and distributed debugging. All the costs, none of the benefits.
The post above rightly calls this problematic, and asks whatever async integration between the services would work, using streams. I’m not quite sure what was meant there. My usual method of integrating different microservices is to not do that, instead. Either we need to send a command to a different service (which is async) or we need to publish some data from a service (also async). Both of these options are assuming to be failure resistant and unreliable. In other words, if I send a command to another service, and I need to handle failure, I setup a timer to let me know to handle not being called back.
Even if you just need some data published from another service, and can use a feature such as RavenDB ETL to share that data. You still need to take into account issues such as network failures causing you to have a laggy view of the data.
This is not an accident.
That is not your data, you have a copy (potentially stale) of published data from another service. You can use that for reference, but you cannot count on it. If you need to rely on that data, you need to send a command to the owning service, which can then make the actual decision.
In short, this is not a trivial matter. Even if the actual implementation can be done pretty easily.
The fact that each service owns a particular portion of the system is a core principle of the microservice architecture.
Having a shared database is like having a back stage pass. It’s great, in theory, but it is also open for abuse. And it will be abused. I can guarantee that with 100% confidence.
If you blur the lines between services, they are no longer independent. Have fun trying to debug why your Users’ login time spiked (Orders’ is running the monthly report). Enjoy breaking the payment processing system (you added a new type of user that the Orders system can’t process). And these are the good parts. I haven’t started to talk about what happens when the Orders service actually attempt to write to the Users’ tables.
The article suggests using DB ACL to control that, but you already having something better. A different database, because it is a different service.
It might be better to think about the situation like joint bank account. It’s reasonable to a have a joint bank account with your spouse. It is not so reasonable to have a joint bank account with Mary from accounting, because that make it easier to direct deposit your payroll. There is separation there for a reason, and yes, that does make things harder.
That’s the point, it is not an accident.
The whole point is that integration between services is going to be hard, so you’ll have less of that, and you’ll have that along very well defined boundaries. That means that we can have proper boundaries and contracts between different areas, which lead us to better modularity, thus allowing easier development, deployment and management.
If that isn’t something you want, that is fine, just don’t go into the microservice architecture. Because a monolith architecture is just fine, but a Frankenstein creation of a microservice architecture with shared database is not. Just ask Mary from accounting…
Spot on. Shared backend database in microservices just means you have a distributed monolith, making your life even harder. Put a very high fence between your domains with very specific API gates.
I am not sure yet if I would call microservices a pattern or anti-pattern by themselves …
architecture erosion? You mean like in geology - the hard rock remains and all the soft and unreliable stuff is eventually gone? Then yes, i think some ideas really need additional protection against reality.
It was the part that many have not been aware of. They crossed the line without even known it.
Microservice architecture is hard, and it necessarily involves a lot of what feels like duplication and repetition to ensure that each component is truly independent and resilient; the Orders service's data includes a non-canonical subset of the User service's data, which is frequently refreshed from the User service by push, pull or a combination of the two; and vice versa.
If you don't have the time, the engineering resources, or the proper incentives to do microservices properly, then you end up taking shortcuts, and the result of doing that is the same as it ever was.
Long time ago, there was SOA, and all was good. People were building autonomous, decoupled, self-contained services communicating with each other asynchronously through a format-agnostic service bus.
Then some nerd with too much time on their hands came up with a new buzzword 'microservices', and the whole industry went bananas trying to figure out how this was different from what they were already doing, and to justify the new buzzword using imagined and manufactured differences between SOA and microservices.
Jay, To be honest, I blame SOAP and the complexity that it dumped on devs and ops.