Over the last few posts, we've looked in detail at Oracle Advanced Queuing. Most recently we looked at "bridges" - or applications that helped us broker messages between normally incompatible protocols like MQTT, AQ, and Oracle Streaming Service (OSS). We built a few bridges, and they worked out great! In this post, I thought I'd present an alternate, possibly more "fun" look at bridges. How might we make it more fun, you say? Well, one way might be to use a solution that doesn't require us to write much code at all. And to do that, we'll use Node-RED! We've talked about Node-RED in the past on this blog (how about installing it in an "always free" VM?) and since it is often used in conjunction with IoT solutions, I thought it would be a perfect tool for building some bridges. 

Before we go on, I should mention that the blog post I linked to in the paragraph above is slightly outdated. I haven't had the chance to update it, but I recently gave a presentation at Node-RED Con Japan about installing Node-RED on an "always free" Arm instance. Here's that session, if you need to get up to speed!

Building an MQTT to AQ Bridge

First, let's look at recreating the MQTT-AQ bridge from this blog post. In that post, we used the following architecture drawing to explain at a high-level the concept of what we were trying to accomplish. In a nutshell, we were connecting the unrelated protocols and passing the messages. A message produced in MQTT is consumed by AQ.

When we look at the "flow" (below) in Node-RED, you can see that it looks similar to the "bridge" portion of the architecture diagram (above).

So how do you build this? Well, since there's no "native" module for AQ in Node-RED yet, we'll have to use the node-red-contrib-oracledb-mod module (blogged here) and enqueue the message directly via PL/SQL. Scary? Nah, no big deal! Make sure you've got the module installed and configured before we proceed. Let's build this flow! Start by dragging an MQTT node in to the flow, double-click it, and enter the topic to listen to.

Now drag in a function node. The function node will let us generate the query and inject the incoming message payload into the query. We'll set the query into the query key of the message object since that's what the Oracle node will be looking for. Don't worry if you can't read the query in the gif below, I'll share it in just a second.

In the function node, we'll use the following to construct the proper query. Of course, you'll need to enter your own queue name!

Finally, add an Oracle node and set it to "ignore the query results" (since the query doesn't generate any output). Since we generated the query in the previous function node, we don't need to do anything else here!

Click deploy.

Once the flow is running, we can connect up in our terminal window to the AQ queue and then publish a message to the AQ topic. If we configured it correctly, the AQ queue should receive the message that was published to MQTT.

Building an MQTT to OSS Bridge

The concept here is identical to the MQTT-AQ bridge. Again, we'll refer to the architecture drawing from earlier in this series. This time we're bridging MQTT with Oracle Streaming Service (OSS). 

Since OSS is compatible with Kafka, we can use the node-red-contrib-rdkafka-secure module to build this bridge. Take note - if you're on MacOS, you'll want to pay attention to the install instructions for that module that refer you to the node-rdkafka instructions so that everything gets installed properly.

Again, the architecture looks almost exactly like the flow.

Like the last flow, this one begins by configuring an MQTT node for the topic we want to listen to.

Next, we drag a secure Kafka out node (configured to point at OSS) to the flow, and connect the two nodes.

We'll also need to configure our Kafka Broker. Refer to the last post if you do not remember how to get this information.

And that's the MQTT-OSS bridge in Node-RED. Deploy it, then test by producing a message to the MQTT topic and observing the OSS stream. Since OSS is Kafka compatible, we can use the kafka-console-consumer.sh script from Kafka to consume our OSS stream.

Summary

We used Node-RED to build our MQTT-AQ and MQTT-OSS bridges. It's a simple, but practical solution to a complex problem. 

That wraps up this short series on messaging and Oracle AQ. I hope you've enjoyed it and learned something along the way. As always, I welcome your feedback on how I can improve future content and thank you for your support.

Image by Mikes-Photography from Pixabay