So far in this series (part 1, part 2), we’ve covered both the “how” and the “why” as it relates to messaging in your modern web applications. We used an e-commerce example to illustrate the need for messaging and we looked in depth at launching a local Kafka broker and creating both an “order” service and a “shipping” service that communicated via the Kafka broker. If you haven’t yet read those two posts, I highly recommend that you do so before proceeding here.

Back from reading them? Great, so you’re all up to speed! The natural next step in this series, I believe, is to talk about how this approach would be applied in your cloud based microservice (or monolithic) deployments. Like always, the approach will heavily depend on your team’s preference. If you have a strong DevOps presence on your team and aren’t afraid of getting your hands dirty then you could certainly choose to roll your own Kafka broker in the cloud by turning up the appropriate VM resources and installing and maintaining them. Other teams might shy away from that approach, and that’s perfectly understandable as it means there are additional resources to maintain (not to mention pay for). 

For those who choose to avoid running their own broker, there is an awesome option available: Oracle Streaming Service (OSS). OSS is a fully managed, scalable, durable and secure service for consuming and producing messages. I know that sounds like marketing speak (probably because I mostly copied it from our marketing page) but in essence, what we are talking about here is a managed service that provides all of the benefits of a Kafka messaging broker without much of the setup and maintenance. Of course, the first objection at this point is usually from those who are afraid of “vendor lock-in” by tightly coupling their application code to a proprietary SDK. That’s a perfectly valid concern, but I would argue that we do this all the time when we consume third-party APIs or use proprietary RDBMS systems, so what’s the difference? Still, if it concerns you I have great news: we can modify our order and shipping services to utilize OSS with nothing more than a few changes to our configuration because OSS offers compatibility with the Kafka API. Yeah, it’s super cool and I’m going to show you how to do it in this post!

Note: I should mention that I’ve already blogged about this topic in depth in a previous post. Therefore, I won’t go into much detail about how to set up your stream or the topics. Please read the previous post for those details.

Switching our Services to use Oracle Streaming Service

In case you missed the note right above this, please refer to this blog post for information on getting your stream and topics configured. Once you’ve gone through those steps and have set an environment variable for your KAFKA_SASL_JAAS_CONFIG you are ready to create a new configuration file at resources/application-oss.yml with the following modifications (update the region in your server URL as necessary):

Now you can run your services with:

And your services will now utilize OSS instead of your local Kafka broker! No, really - that’s all it takes! After you place a few “orders” locally, you can confirm that the services work just as they did before. You can also verify the messages via the Oracle Cloud console if you’d like.

My order-topic:

My shipping-topic:

Summary

In this post, we discussed options for messaging in the cloud and looked at why using Oracle Streaming Service is a valid and smart choice. We then swapped out our local Kafka broker for OSS in our previous e-commerce order and shipping microservices. 

Check out the code used in this post on GitHub:

• https://github.com/recursivecodes/shipping-svc-kafka

• https://github.com/recursivecodes/order-svc-kafka

Image by 춘성 강 from Pixabay