Derick Bailey is a good friend of mine. You may know him from the Entreprogrammers Podcast or if you listen to Get Up and CODE, he has been on as my guest a few times.
Derick is a software developer by trade. He is also an entrepreneur and has launched a number of notable products and services. He is the creator of the widely popular Marionette.js framework.
Many, if not most, software developers use messaging but don't really know that they are using it. I thought it was a good idea to bring a guest expert on to talk about messaging and message queues. It was a great opportunity to interview Derick on this topic. He has been a professional software developer since the late 90’s, and has been writing code since the late 80’s. He has worked with messaging patterns over the years and was happy to share his knowledge to everyone.
Please check out the video interview with Derick here. And below is the Q&A I asked him to put together to share on the site, about RabbitMQ and other message queues.
What is RabbitMQ? Why would I want to use it?
RabbitMQ is a message broker and message queueing system. It allows you to write separate applications and have them communicate with each other without directly knowing one another. You send messages to RabbitMQ; it puts them in a queue (based on your settings and configuration); and another code picks up the message from the queue to run that code and process the message.
Okay, RabbitMQ is all about messaging. So what are are the benefits?
There are a lot of benefits to messaging, including decoupled application and system design. You can replace parts of your system with new parts relatively easily. You can have multiple programming languages, like C#, Java, Node.js, PHP, ruby, and more, all communicating with RabbitMQ (or any other messaging server). You can use it to integrate legacy systems as well. Fault tolerance is built in to your systems, and you can reduce response time for web requests in some cases.
There are other benefits as well, but these are the main ones I've noted.
How can RabbitMQ reduce response time for HTTP request handlers?
Rather than having complex and time-consuming code in a web server that forces users to wait with a spinning buffer GIF, you get to let your back end code crash when there's a problem and then retry later when the problem is fixed. Since the user doesn't know about the back end code, they don't care that anything broke. The web server still works. Then fix the back end code and reprocess the message.
Do you have a concrete example of using RabbitMQ?
One of the easiest examples is sending an email from a web server. Say you have a contact form on your site. Someone fills it out and expects it will be sent to you via email. Instead of having the web server send the email directly, you publish a message to RabbitMQ. Some other code picks up that message and sends the email.
The web server can very quickly respond to the user and say, “Hey, I sent that email you wanted to send… it's all good!” Meanwhile, on the back end, the code that sends the email is sitting in another process somewhere. It can check to see if the SMTP server is up. It can take its time formatting the email. It can even handle the SMTP server being down and retry the email send later. And the user doesn't have to wait around for it. The user was told the email was sent… and it will be sent as soon as the back end code has everything it needs.
You mentioned “other” message queues. What else is out there?
There are a lot of messaging servers and services available. Some I am at least familiar with by name include:
- ActiveMQ
- ZeroMQ
- Websphere MQ
- MSMQ
- Amazon's SQS
- Azure ServiceBus
There are others still, but these are the ones that I can name off the top of my head. You can also use things like Redis as a basic pub/sub service, which simulates queueing.
Redis as a queue? How does that compare to RabbitMQ?
Using Redis as a queue gives you basic pub/sub capabilities with tools like “resque” for ruby or “kue” node.js. These libraries will provide a first-in-first-out ordering of messages, like a proper message queue and broker.
The main difference between using something like Redis versus RabbitMQ is having a do-it-yourself kit instead of purpose-built tools with a more refined feature set.
Imagine going to a hardware store and buying the plans for a cabinet, then buying the pre-cut wood for it, along with some extras and add-ons that you want in your cabinet. This works. It's great. It also lets you build a small bench because you had some leftover wood. This is the Redis version of queueing.
Now go to a specialized cabinet store. At this store, they have cabinet styles that you would never be able to build on your own. They require more work and knowledge of carpentry, very specialized tools, and sometimes a team of people to build these cabinets. You can buy some very fancy, very configurable, and very nice cabinets at this store. But you don't have to build them yourself. You just tell the store what you need and they configure and build it. This is RabbitMQ—purpose-built, extensible, feature-rich, and offering services that can't be done with the do-it-yourself approach.
What about other queues, like ZeroMQ? How does RabbitMQ compare to that?
ZeroMQ is, by default, a brokerless queueing system. Think of it more like peer-to-peer messaging, where you have to know about the system on the other end. This is similar to how AJAX requests work on the web, too. The web browser knows about the website. When the browser makes an AJAX request, it is sending the request to a specific URL for the site. ZeroMQ works in a similar way. There is no broker in the middle to handle the message distribution.
RabbitMQ, on the other hand, is a message broker. It lets you have two or more systems talk to each other without having to know about each other at all. Each of the systems needs to know about RabbitMQ, but that's it. This has the advantage of decoupling the systems from one another. They don't know about each other at all, but they can still communicate.
Broker vs. brokerless is not “good vs. bad” by any means. They are both valid and valuable means of messaging. They each have advantages and disadvantages. Personally, I prefer the brokered model.
With all the flexibility and advantages, is messaging a golden hammer turning everything into a nail?
Certainly not! There are no golden hammers or cure-alls for anything in the world.
The advantages do come with disadvantages in queueing. By creating a decoupled system of applications that use messaging, you lose the direct flow of code. You now have to look through the messaging patterns to understand how things work, and that can be confusing. There are management and monitoring tools that can help, too.
But there are plenty of scenarios where RabbitMQ or messaging is just a bad idea. If your website needs to read data from a database, for example, this is probably a bad place for RabbitMQ. Sure, you can use Request/Response patterns to make it work… but if the web server is allowed to connect to the database anyways, then just go straight there.
Sometimes it does make sense to have RabbitMQ involved with database work. But sometimes it doesn't. It’s a large gray area that can be complicated.
What are some good ways to learn these complicated gray areas?
A lot of resources exist around the internet for learning RabbitMQ and messaging in general. As you begin using RabbitMQ and building systems, you'll start to get a sense of how things work through trial and error. But I do recommend investing in some of these resources:
- Enterprise Integration Patterns – the book on messaging patterns
- RabbitMQ In Action – a great book for learning the basics and using as a reference for RabbitMQ
- The RabbitMQ tutorials – at RabbitMQ.com, these basic tutorials will show you some common patterns
- My RabbitMQ For Developers package – screencasts, ebooks, and interviews with experts to quickly get up and running with RabbitMQ and node.js
Okay, last question: Why is John Sonmez so handsome?
… Really? O_o :D
