As I’ve stated, I’m willing to eat crow on my path to success. This post reflects what I have unfortunately learned the hard way. In some of our applications we have used messaging in a Send / Reply pattern using messages like FindAddressesForCustomerMessage and FindAddressesForCustomerResponseMessage. In the beginning this worked just fine. However, after we began to grow and scale became more and more of a concern this began to cause us a lot of pain. I’ll state below the pain points of using a service bus / messaging in this manner.
To understand the needs of our system we have ~10 application / web servers and ~1000 users of our CRM system. Collectively they all participate in service bus communication.
Messaging with any of the .NET service bus projects has many things to prevent failure of any kind, can withstand restarts, failover, load balancing, transactional message processing, etc. This is great if you take a step back for a moment to consider where this is important. Capturing the commands and events of users into a persisted store somewhere. When a user does something we’re pretty dang sure it will be successful. No longer needing to display the message to our user, sorry this failed please try again later.
Using messaging for queries forces an asynchronous pattern when it doesn’t necessarily need to be, making the system much more complex to work with. This doesn’t mean you shouldn’t use messaging at all. I could see using a message when the application starts up that has a response used to populate a list of users / password hashes in a local cache for example. This doesn’t mean that the query side is asynchronous though it just means that it looks to a cache for the information and has a dependency that a response is returned before someone can successfully logon.
The continual retry on failure for messaging implementations causes problems when user endpoints aren’t online. For example, we had a shift that would leave and the system would slow down. Why? The outgoing messages would retry over time up to as many as 2 days causing bottlenecks for legitimate messages that needed to get through.
Messaging doesn’t handle large messages very well. Let’s face it sometimes messages for queries aren’t simple things that are composed of 3 or 4 fields. This forces us to work around the size limitations put in place for all messaging solutions.
Can these things be overcome? Sure, I could invest my time heavily in addressing my concerns as they relates to the service bus code-base. However, my needs might not be compatible with the needs over others and therefore having to maintain my own version. It’s okay to admit when we were wrong in our assumptions. You can disagree with me and tell me about all of the various changes or tweaks that could be made. However, all I will be thinking is that you haven’t actually tried this for yourself. Next time it won’t be me eating crow.
So what is the right solution? My recommendation is to spend some time reading and learning about CQRS (Command Query Responsibility Segregation) and pay attention to the distinct separation of queries and commands.
Thanks for posting your thoughts on this. Sort of a follow-up to our discussion after the user group meeting the other night. I think this is an important skill to learn, the ability to critically reassess your assumptions from time to time and to not blindly use technologies/patterns just because it helped so much in the last project context. It can really be a detriment to each new project if you always bring in the same, heavyweight, enterprise tools and frameworks every time, especially when the scope of the project might be small or the tools not the right fit for the new context. I suppose you should always try to get a fresh new perspective and base set of assumptions with each new project.
ReplyDelete