How to Implement RabbitMQ with NodeJS and AWS SES

Hello everyone, this will be my first medium article, today I will talk about the logic of the queue structure using RabbitMQ with NodeJs and what it is used for. Before moving on to RabbitMQ and its example in Nodejs, let us first understand what is a message broker.

Message Broker

Message Broker is the general name given to message queue systems. Its purpose is to ensure that a message received from any source is transmitted to another source via message protocols as soon as it is due. RabbitMQ, Kafka, MsMQ, Azure Service Bus, ActiveMQ are widely used message broker software. Now let’s talk about what the RabbitMQ which is our main subject😎

RabbitMQ

RabbitMQ is an Open Source message system. It is based on keeping the data in the queue and then processing the required queue by means of the relevant queue listeners in order to avoid the problems that may occur during the processing of a large number of data and the delayed response time to the user, especially in systems with intense data flow. RabbitMQ can be a good choice because these processes are performed asynchronously, because the running application is not kept in vain and the transaction cost on the server is minimized.

Why should we use RabbitMQ

Let’s assume that we need the following in the system where we will use RabbitMq: For example, we want to send a welcome mail to users who are registered in our system or the scenarios such as we want to keep log records of user operations in the system into the our database,In such scenarios, if we need to activate processes that are not instantaneous and take a long time as a result of the operation,We need to reduce the application density by process these processes asynchronously. Otherwise, the user will be exposed to an unnecessary Response Time and there will be a situation against our application.

Implementing RabbitMQ with NodeJS

Before we start the demo application, I just want to mention some terms which we will use in implementation.

.Publisher : Application that send messages to the queue.

.Consumer : Application that listen to the queue

.Queue : The queue to which messages are added with RabbitMQ

.Exchange : The messages generated by the publisher arrive at the broker. The exchange is the first to meet the message after the message has been received. Exchange sends the message to the corresponding queue. If our data need to be manipulated we do this process here.

.FIFO : Messages exit the queue with FIFO (First in,First out) logic

After we learn these kind of terms now we can start 😋

I skip the express in nodejs server setup and routing part because our main goal is RabbitMQ. If you don’t know how to setup an express server you can check it here. In this demo we get learn how to use Rabbitmq cloud server with nodejs.

• First of all we need CloudAMQP account, you can create a new account from that url.
• After creating a new account and you login you see this page.

Main Page of CloudAMQP

In this page, you see your instances if you create before. You can see my “Test” instances I create before, to create a new instances click on the “Create New Instances” button.

Creating new Instance page

In this page we start to create our new instance,

• Write a name for your instance to Name input area.
• We select the Free plan to test RabbitMQ.
• You can add some tags to separate your instances between projects.
• After fill these input area click the “Select Region” button.

Selection ragion page

• Select a AWS data center for you and click “Review” button.

Confirm page

• In this page you can see your new instance informations. If you don’t want to change anything click “Create instance” button.

Main Page of CloudAMQP

• And Well Done! You create your first instance successfuly.
• Instance details such as connection URL, server name, user/vhost, and password can be found on the details page. To see that page click on your instance name.

Instance Detail Page

• From this screen you can access your amqp url information, we will use this url string to connect to the remote server that RabbitMQ has prepared for us.

Producer

producer.js

• First of all we need to install amqplib packages using npm with this console comment => npm install amqplib
• In the first line we need to require the necessary library first.
• In the second line we initialize .env config to hide my AWS configurations and RabbitMQ cloud url for security.
• At fifth line export a helper function to send message to queue. It takes a parameter called queuName which represents the name of the queue to which the message will be sent and data represent the object that has information about the user.
• In order to use RabbitMQ service in the Cloud via the application, we must first access the relevant service. We connect to RabbitMQ server by using connection string we talk about it previous part (Line 6)
• If there is an error while connecting to the remote server, we log this error to the console.
• After we succesfully connect our server then we need to open a channel to create the queue we will use.
• We are connecting the queue with the channel we have created.(Line 15).
• Sending the message to the queue (Line 22)
• 1000ms after adding the message to the queue, we close the connection.

Consumer

consumer.js

• All the steps are the same with producer part up to line 28, In that line, we specify the queue the channel will consume. After that, parsing data via JSON.parse method and we send that data object to mailService’s emailService function as a parameter. We will talk about emailService function in next part.
• After the consuming process is successful, we set the relevant message as “noAck:true” to remove the message from the queue.

AWS-SES Service

aws-ses.js

I will not focus on the use of Amazon SES Service, but the main goal is intended to be applied here is :

• When a user send a feedback to us, we send that user a thank you email and also send user’s feedback email to the our admin.
• My email service function takes receivedEmail(string) and userInfo(object) as a parameter.

If you want to get more knowledge about AWS SES you can check documention.

Database Schema

feedback.model.js

Route

feedback.route.js

Controller

• In controller we take user’s feedback and save it into database(line7)
• After saving data we call producer and set queue name as “feedbackQueue”, also send user’s information by string(line9)

Testing

• Start the consumer to listen feedbackQueue and nodejs server.

• Sending Post request with Postman

• After Sending post request our producer send the message to queue after that consumer call the AWS SES service and our mails successfuly sended.

• User’s feedback record successfuly saved in our database.

User gets the email from admin

Admin gets the email which has user’s feedback

On this page we can see which channel used our server.

On this page we can see the our feedback queue which set by us.

Also we can see that after consuming message remove the queue, let’s improve this information

• I only run our nodejs server but not consumer and again send another post request

See there is a message to consume, now I run my consumer console app in terminal.

My consumer start to listen the feedbackQueue and it consume the message ready to consume. Again if you check your RabbitMQ cloud panel you see ready message is 0.

Conclusion

RabbitMq one of the most populer message broker service. In this article, I shared a RabbitMQ demo project integrated with a Amazon Simle Email Web Service in order to understand RabbitMQ working logic.

I hope you enjoyed with this article and demo application. If you want to see full code of this project you can check my github repo.

Thanks for reading 😊