We've been on a long and interesting journey with microservices in this blog series and have covered a lot of topics. In this post we'll take a look at something slightly different - storing JSON documents. There are tons of options when it comes to storing JSON document collections - some more popular than others. I'm certainly not here to discuss the merits of other options or discount their popularity, rather I'd like to show you an alternative to the more popular options and present a few reasons why it might be beneficial to your application to consider them.We'll look at using your Oracle Autonomous DB instance to store JSON documents using the Simple Oracle Data Access (SODA) APIs. SODA is a set of NoSQL-like APIs that let you persist, retrieve and query JSON document collections with Oracle DB. There are several implementations available to choose from: Java, Node.JS, Python, C, and PL/SQL - as well as a REST implementation that can be used with any language/platform. For a change of pace, we'll use the Node SDK in this example.

If you haven't been following this series you might want to consider reading the first few posts to get up to speed if you plan on trying this example:

Intro posts:

• Intro
• Getting Started With Kubernetes And Docker
• Getting Started With Autonomous DB

Getting Started

In this post we're going to build a "post" service for a simple social media type service that allows users to submit content (text, links, images, videos) to the application. The "user" service that would be deployed with this application could be either of the user services that we created in the previous posts. Links to those blog posts can be found at the bottom of this post if you want to check those out. We'll use the ExpressJS framework to make things easy. Start out by scaffolding the application with the express CLI:

express --no-view post-svc 

This gives you a basic application structure to get started with. Let's grab some dependencies next. Run the following command to install them (we'll cover what some of them are used for later on):

npm install @hapi/joi aws-sdk cors express-async-handler express-fileupload uuid oracledb

Yeah, you read that right. We'll be using the aws-sdk within our application even though we're deploying to the Oracle Cloud. Since there isn't currently an Oracle Cloud SDK for Node, we'll take advantage of the fact that Oracle Cloud's object storage service exposes a fully compatible S3 endpoint and use the AWS SDK to upload objects. More on that later on.

We won't need to create any tables for our JSON document collection, but we will need to create a schema/user. Run the following as an admin user to do that (note the special 'soda_app' privilege we're granting here):

We're going to need our Oracle ATP wallet (check this post for how to generate/download the wallet). Create a directory in the project root called build-resource/ and place a copy of your ATP wallet at build-recource/wallet/. You'll also want to create an object storage user and bucket in your Oracle Cloud tenancy and generate an access token and secret key. We'll need to set some environment variables, so I like to create a file called env.sh in my build-resource directory to handle setting all of them so I can source that file when working with the project. That file usually looks like this:

Source the env.sh file to set these variables into your session.

To test and deploy our application we're going to utilize Docker. This will keep everything nice and compartmentalized in the container and prevent us from installing dependencies on our local machine. The Dockerfile is not complicated, so create one in the root that looks like so:

We're using Oracle Linux for the base image, installing some dependencies (notably the Oracle DB instant client), copying our wallet into the image and our application source code, installing our app with NPM and then starting the app in our entry point. At this point we can build and run the application to make sure we've got everything set up properly:

Confirm that the application responds at http://localhost:3000.

Now let's rename the stock 'user' routes that Express gives us in the routes/ directory to posts.js (make sure to update any other references in the application, such as in the app.js file). The posts.js route file will be where all of our endpoints are defined for the post service.

The Post Service

Before we define our endpoints, let's create a service that we will use for our persistence and query operations. Create a new directory called service/ in the root of the project and create a file within that directory called post-service.js.  The post service will be a class that creates our default connection pool and performs our database operations. Start out by importing some dependencies, setting some options on the oracledb object and creating an init() method to handle the connection pool creation:

Now modify app.js to create our service object and set it into the application so that we can retrieve it later on from our route controller. I'll include the whole file contents here, but take note of the additions we made on lines 5, 25-27 and 30-33.

Now let's add a save() method. Here's where we'll see the simplicity of SODA in action. The save method accepts a JavaScript Object, grabs a connection to Oracle DB, and grabs the 'soda' database object. We use the soda object createCollection() method which will create the collection in the database if none exists, or return an existing collection by the name we specify - either way, we get the collection back that can be used going forward. To persist our 'post' JS object, we simply call postCollection.insertOneAndGet() which does just what it says on the tin: inserts the object and returns it to us. That's it. Our JSON document is stored in our collection.

Updating is almost identical to saving a new object, the only difference being that we first retrieve the object and then call replaceOneAndGet():

To get a post by ID, we use find().key(id).getOne():

To delete a post by ID, we use find().key(id).remove():

We can also query by example to find posts by elements contained within the JSON document itself. For example, we can query by the userId key within our post JSON like so:

The full documentation for using SODA with Node contains many more methods that can be used in addition to those shown above.

Object Storage Service

With our CRUD methods implemented, we can now create an object-service.js class that we'll use to upload items to object storage in our application. As I stated earlier, we're using the AWS SDK for this operation:

And that's all we need to do to upload objects to our Oracle Cloud object storage bucket. We could expand this service to implement other features as needed, simply refer to the AWS Node documentation for the necessary methods.

Validation

Now let's add some validation for our posts objects because even though we're using JSON document storage, we'd still like to implement some validation rules on the objects that we are persisting. Create a file called model/post-schema.js:

We're using Joi from hapi.js for our validation, and we'll call this schema in our route controller to ensure our post objects are valid before persisting.

Updating The Router

Head back to our routes/post.js file and add our endpoints that will call our post service. This is a pretty standard REST implementation with the only notable items being the file uploads that use the object service if a file has been uploaded and the addition of the cors() and asyncHandler() middleware

Testing Endpoints

At this point we are ready to test our endpoints. Build and run the Docker container and use test them out via cURL:

This service will persist social media "posts" as JSON documents. The general document format is as follows:

Save a new post (image/video - returns `201 Created`):

Save a new post (text/link - returns `201 Created`):

Save a new post with invalid data (returns `400 Bad Request`):

Update an existing post (returns `200 OK`:

Update an existing post with invalid data (returns `400 Bad Request`):

Get a post by ID:

Get all posts by user ID:

Get posts by user ID (paginated):

Delete a post:

Delete a post that does not exist:

Deploying

You can push this service to your OCIR Docker Registry:

Refer to this app.yaml file for an example that can be used to deploy to your OKE Kubernetes cluster.

Summary

In this post we created a microservice that persists JSON documents in a collection within our Autonomous Transaction Processing (ATP) database in Oracle Cloud. We added support for object upload in that service and deployed it in a Docker container on Kubernetes. 

Reference

If you'd like to catch up on the previous posts in this series, please refer to the links below.

Helidon And Hibernate:

• Building A Helidon Microservice Part 1
• Building A Helidon Microservice Part 2
• Building A Helidon Microservice Part 3

ORDS With Micronaut:

• Microservices The Easy Way With ORDS And Micronaut - Part 1
• Microservices The Easy Way With ORDS And Micronaut - Part 2
• Microservices The Easy Way With ORDS And Micronaut - Part 3

The full source for this example can be found on GitHub: https://github.com/cloud-native-microservices/post-svc-node-soda 

Photo by Wade Austin Ellis on Unsplash