This article is part of a larger series. To view the article published prior to this one, click here.
Before we get into the details of messaging and the differences between various protocols and implementations we should cover some basic workflows. However, before we can get into that, we need to examine the basic components that comprise all messaging. So we’re going to examine that abstractly in this article.
First, is the message. All messages are simply collections of key/value pairs. If you know messaging, this might sound like a radical simplification. Bear with me. I hope my simplification will make sense shortly. Commonly, a message is described as being comprised of headers and a body. As I see it, what is the body but simply another header that stores the message body as its value. By viewing the body as just another header, I feel this helps simplify an analysis. Removing a needless extra classification differentiating headers from the body means we only need to concern ourselves with headers. Thus, we can define general best practices for messages without having to clarify if they apply to headers, the body, or headers and body. You should see how this helps as we go through our analysis.
So, messages are simply collections of key/value pairs.
A Producer will put together a collection of headers and send it to the Broker. How these headers are compiled and sent to the Broker depends on the messaging protocol. We’ll save that for when we compare different messaging protocols. To simplify things, we’ll use Java Message Service (JMS) in our code examples until we get to those comparisons.
The Broker receives the message and holds it until the Consumer requests a message. Most protocols provide some way for a Broker to notify a Consumer that a new message is available. Again, since this varies across protocols, we’ll save this for the comparisons. An important point is that the Broker simply holds the message. Treating the message as immutable. This is an important part of the duty of a Message Broker that is often over looked. If a Message Broker alters the messages, it would be an Enterprise Service Bus (ESB). In reality, aa ESB is just a Message Broker with extra applications that consume messages to process them in some way and then produces messages with the new content. The messages themselves in an ESB aren’t actually being changed.
With the message on the queue in the Broker, it is available to be served. Many messaging technologies have some way to notify a Consumer that a message is available. This is commonly called Subscription. But for now, we’ll focus on the basics: a Consumer requesting the next message from the Broker. I’d like to highlight my phrasing of “requesting the next message”. It’s important to remember that a Message Broker isn’t a Key/Value, Store, or a Database. A Message Broker’s job is to store messages in a Queue. The defining trait of a Queue is that it stores its contents in the order they were received to be served in that same order. When a Consumer requests a message all it’s doing it requesting the next message. When a Consumer requests a message the Broker sends a copy and effectively checks the message out of the queue.
This is an important part of the process. The Broker holds a copy of the message until the Consumer sends a request to delete the message from the Broker. This is commonly called an Acknowledgement.
Once the Broker receives this Acknowledgement it deletes the message.
This is an important part of Messaging because this is how to ensure reliable message delivery. Consider a failure in the Consumer. If the Broker had deleted the message when the Consumer requested it, and didn’t wait for an Acknowledgement, then what if the Consumer failed? If this happened, the message would be lost when the Consumer failed, and the Broker would have already deleted the message.
This why handling Acknowledgements properly is a critical part of creating a reliable Messaging workflow. Other systems like Databases use Transactions to ensure reliability. This is because there is nothing inherent in those protocols to allow for these sort of confirmations. So we layer Transactions on top of them, but in Messaging we already have such safety measures with Acknowledgements. This allows us to be deliberate with each step in our workflow without having to incur the cost of atomicity, or isolation just to be consistent.