RabbitMQ : Tackling the broker SPOF

Though RabbitMQ brokers are extremely stable, a crash is always possible. Losing an instance altogether due to a virtual instance glitch is a likely possibility that can't be ignored. Therefore, it is essential to tackle the broker single point of failure (SPOF) before something bad happens, to prevent losing data or annoying users.The RabbitMQ can easily be configured to run in an active/active deployment, where several brokers are engaged in a cluster to act as a single highly-available AMQP middleware. The active/active aspect is essential, because it means that no manual fail-over operation is needed if one broker goes down.

Mirroring queues

With clustering, the configuration gets synchronized across all RabbitMQ nodes. This means that clients can now connect to one node or the other and find the exchanges and queues they're expecting. However, there is one thing that is not carried over the cluster by default: the messages themselves. By default, queue data is local to a particular node. When a queue is mirrored, its instances across the network organize themselves around one master and several slaves. All interaction (message queuing and dequeuing) happens with the master; the slaves receive the updates via synchronization over the cluster. If you interact with a node that hosts a slave queue, the interaction would actually be forwarded across the cluster to the master and then synchronized back to the slave.

Activating queue mirroring is done via a policy that is applied to each queue concerned.

Federating brokers

If you think beyond the notion of a single centralized enterprise resource and instead think in terms of distributed components, the idea of creating a topology of RabbitMQ brokers will emerge. RabbitMQ offers the following two plugins that allow the connection of brokers:

• The shovel plugin, which connects queues in one broker to exchanges in another broker

• The federation plugin, which connects queues to queues or exchanges to exchanges across brokers 

Both plugins ensure a reliable delivery of messages across brokers; if messages can't be routed to the target broker, they'll remain safely accumulated. Neither require brokers to be clustered, which simplifies setup and management (RabbitMQ and Erlang versions can mismatch). The federation plugin needs to be installed on all RabbitMQ nodes that will engage in the topology.

Keep in mind...

• RabbitMQ relies on Erlang's clustering feature, which allows several Erlang nodes to communicate with each other locally or over the network.
• The Erlang cluster requires a so-called security cookie as a means of cross-node authentication. 
• If RabbitMQ instances are firewalled from each other, its needed to open specific ports on top of the one used by AMQP (5672); otherwise, the cluster will not work.
• Make sure the same version of Erlang is used by all the RabbitMQ nodes that engage in a cluster; otherwise, the join_cluster command will fail with an OTP version mismatch error.
• Similarly, the same major/minor version of RabbitMQ should be used across nodes, but patch versions can differ; this means that versions 3.2.1 and 3.2.0 can be used in the same cluster, but not 3.2.1 and 3.1.0.
• In a federation, only the node where messages converge needs to be manually configured; its upstream nodes get automatically configured for the topology. Conversely with shovels, each source node needs to be manually configured to send to a destination node, which itself is unaware of the fact that it's engaged in a particular topology.