NSQ Realtime Distributed Messaging

NSQ is a real-time distributed messaging platform written in Go and created by well known service bit.ly.

It is simple and easy to use and has a good GUI of admin. NSQ is one of the best option if you want to implement Message Queue.

Concept of Message Queue

Message Queue is an implementation of the publisher/subscriber architectural pattern which is used for communication between different parts (applications, services, etc) of your system.

 

Message Queue process flow

 

In short, when an event happens by a message published (producer) to the Queue (shown as picture above).  Any services that interested in that services will subscribe (consumer) to get the message from the Queue.

NSQ Component

NSQ components consist of nsqd, nsqlookupd, nsqadmin.

nsqd is the daemon that receives, queues, and delivers messages to clients.

nsqlookupd is the daemon that manages topology information. Clients query nsqlookupd to discover nsqd producers for a specific topic and nsqd nodes broadcasts topic and channel information

nsqadmin is a Web UI to view aggregated cluster stats in realtime and perform various administrative tasks.

 

 nsqadmin GUI

 

Download

First of all, you need to download the binary here as following your operating system and in my case, I use Windows then I download binary for Windows.

Open extracted folder and there you can see different executables:

• nsqlookupd.exe
• nsqd.exe
• nsqadmin.exe
• .. and many others 

Step 1. Run nsqlookupd

Open extracted directory in shell/command terminal you prefer and run

C:\nsq-1.2.1.windows-amd64.go1.16.6\bin>nsqlookupd.exe

You should see the following output as below

nsqlookupd

 

 

This picture says that nsqlookupd is running and has two interfaces:
one is using TCP with port 4160 and another is using HTTP with port 4161

 

 Let's check the result

Check with browser http://localhost:4161/topics

 

Check with curl localhost:4161/topics

 

 

 This is the result you should get and its fine since there is no topics registered yet

 

 

Step 2. Run nsqd

Next step, run nsqd in terminal

 C:\nsq-1.2.1.windows-amd64.go1.16.6\bin>nsqd --lookupd-tcp-address=127.0.0.1:4160

 

 

 You should get result as shown as picture below

nsqd in terminal

 

 

Step 3. Publish a Message

Let us publish our first message in Queue. Open terminal to execute curl:

 

curl -H "Content-Type: application/json" -X POST http://localhost:4151/pub?topic=test -d "{\"Hello World 1\"}"

If the request is 200 OK, our new topic will be created automatically

 

  curl in terminal

 

 

Step 4. Run nsqadmin

 Let's check the message in admin UI. Open terminal to execute nsqadmin

 

 C:\nsq-1.2.1.windows-amd64.go1.16.6\bin>nsqadmin --lookupd-http-address=127.0.0.1:4161

 

 

Result in browser http://localhost:4171

 

 

 Click in topic to show detail

 

 

 

Step 5. Alternative to check with nsq_to_file

Open in terminal to execute nsq_to_file

C:\nsq-1.2.1.windows-amd64.go1.16.6\bin>nsq_to_file --topic=test --output-dir=/tmp --lookupd-http-address=127.0.0.1:4161

 

 nsq_to_file in terminal

 

 

 nsq_to_file will create new file test.LAPTOP-91JM58QI.2023-02-28_16.log in folder C:\tmp

 

 

 

File created by nsq_to_file

 

 

Try to send message as below

 

curl -H "Content-Type: application/json" -X POST http://localhost:4151/pub?topic=test -d "{\"Hello World 1\"}" 

 

curl -H "Content-Type: application/json" -X POST http://localhost:4151/pub?topic=test -d "{\"Hello World 2\"}" 

 

curl -H "Content-Type: application/json" -X POST http://localhost:4151/pub?topic=test -d "{\"Hello World 3\"}"

 

 

 

send Hello World

 

 

 If you open the file test.LAPTOP-91JM58QI.2023-02-28_16.log in notepad then you will get message

 

{"Hello World 1"}
{"Hello World 2"}
{"Hello World 3"}

 

 

 

I think that's all. If you read something interesting from this article, please like and comment below. Thank you all ... Happy coding!