Pub/Sub is a message-driven software design pattern that facilitates communication between complex system components through an efficient communication mechanism that utilises message passing. In a typical pub/sub system, there is the producer (publisher) who creates a message to be sent out to Subscribers.
The Producers do not explicitly fashion the messages for specific consumers, but instead, messages in the system are grouped into categories and the consumers indicate interest in a specific group of messages without caring about the creators of the messages as seen in the image below.
In a typical distributed real-time system setting where each component of the system performs a specific task without knowing about other parts of the system, pub/sub comes in handy because it decreases communication complexity for the parties involved. A component generates a message at the end of its task and passes it to the appropriate channel. The next component can receive the message and Initiate the next needed task and so on.
How pub/sub works
The pub/sub-system consists of four essential components which play unique roles in how the system works. Below are the roles of each component.
Publishers: Publishers, also called producers, are any part of the system that produces data. The publishers create and propagate messages through a channel to the interested receivers without any knowledge of the receiving party. This implies that the message they send is not fashioned specifically for a specific receiver since they do possess any knowledge of the interested subscribers or whether any subscriber is interested.
Subscribers: Subscribers are any part of the system that consumes data. They retrieve messages when available and perform operations based on the information. In a typical pub/sub system, a single receiver can subscribe to many channels.
Channels: The channel is the data stream through which data is transmitted from the publishers to the subscribers.
Messages: The messages are data being sent through the channels. The message sometimes comprises the header(metadata) and the body of the payload.
In the pub/sub system, a publisher typically publishes a message to a channel and the subscribers of the channel receive the message and perform the needed operation(s) on them. But the unique thing about the pub/sub system is that the identities of the parties involved are not essential and therefore can be left out.
Also, the publisher of the system can also serve as the subscriber and this makes the whole system much more flexible.
The channels on the other hand can be viewed as categories or a stack of similar items which in this case would imply that message propagation is the process of pushing new or similar item(s) to the stack. Going with the stack analogy, this implies that messages sent through the channel are always guaranteed to arrive in the same order.
It is worth noting that in order for a subscriber to receive a message they must have subscribed to the channels before the message is sent. This is in-line with all message-driven designs, a subscription has to be made before the message is received.
Pub/sub frameworks
The following are several well-known frameworks that support pub/sub:
Redis: Redis is an in-memory data structure stored as a database, cache, message broker, and streaming engine but it also comes strapped with a pub/sub system of its own.
Apache Kafka: is a popular and robust messaging tool that has the usual Pub/Sub features. Apache Kafka is also designed to include a message log. Subscribers can step back and forth in time to ‘replay’ messages.
RabbitMQ: is a message queue that does not really follow the pub/sub pattern. However, it can be configured for direct message exchange between two or more components of the system.
Importance of pub/sub in real-time messaging
- Ease of communication: With the introduction of pub/sub into a real-time system, all components of the system benefit from the ease of communication it brings as the components can communicate in an asynchronous way.
- Loose Coupling Between System Components: For a real-time system where each part performs a special task of its own, pub/sub fit in well and benefit from loose-coupling. This further strengthens the system as each part can be tested independently without depending on the other components.
- Frugality With Resource Consumption: With pub/sub, components of the system do not need to poll each other for updates but can instead rely on the channels for updates. This reduces the consumption of network bandwidth as well as other resources involved.
- Guaranteed Consistent Reliability: Since communication is facilitated by message passing, modification of one component of the system can be done without affecting other parts of the system as long as the message they publish is valid.
- Scalability: The introduction of pub/subs into a system allows the system to be scaled without the fear of breaking functionality because both the subscriber and consumers are separate entities and function independently.