Wednesday, April 6, 2016

Create an oracledb enabled Node.js application container

In my previous article, First setup of a connection from Node.js to an Oracle Database, I demonstrated how to make a connection to a remote database using Node.js and the node-oracledb module. I used a dedicated VM with Linux 7 installed and Oracle Instantclient provided the 12.1 client libraries.

Now it's time to take it a step further.
Let's create an application container and just start it multiple time running any .js script and be able to connect to an Oracle database.

I will start by demonstrating you how to manually build a Docker image with Node.js and the node-oracledb module. This image can then be used to launch as many application containers as you like. (Depending on your resources off course)

For this setup I have installed Docker on an Oracle Linux 7 VM in VirtualBox.

Create the Docker image manually


Create a Linux base image


Logon as root (or use sudo) on the Oracle Linux VM.

First we need an operating system for the container.

Pull docker image of oraclelinux from the Docker hub.

docker pull oraclelinux



Now start an interactive Docker container.
docker run -ti oraclelinux /bin/bash



Within the container we will creating a non-privileged user and install the required OS packages (including dependencies).
The user can be used to run Node.js scripts in the container without root privileges.
useradd nodejs -p '$6$salt$ZjJzVKp5xtoIl7cfXqZe0mQjWeOpsV2pMiIYpWzkR4ExCBpPdT3mi3eXtG1MSawJnZfXFjBcq0UUmenLq1Cj//'



note. I used python to created the encrypted password I used when creating the os user. For your convenience the command:
python -c 'import crypt; print crypt.crypt("Welcome01", "$6$salt$")'

Install the required OS packages including dependencies

yum -y install unzip libaio gcc-c++ tar make curl


Create the base image


Exit the container and commit the container to create a base image.
exit
docker ps -a
docker commit 51ce97aa511f


Tag the image to give it a name and version, linux-base:1.0
docker images
docker tag 19de63788941 linux-base:1.0
docker images


Install Oracle Instantclient, Node.js and the node-oracledb module


Now that we have a base image, we are going to run a new container based on this image.
I have downloaded the Oracle Instantclient from the OTN site and put them in the /tmp/Downloads directory.

instantclient-basic-linux.x64-12.1.0.2.0.zip from Oracle OTN
instantclient-sdk-linux.x64-12.1.0.2.0.zip from Oracle OTN

Start an interactive container using the created linux-base image and share the /tmp/Downloads directory using a volume in Docker.
docker run -ti -v /tmp/Downloads:/tmp/Downloads linux-base:1.0 /bin/bash

Install Oracle Instantclient

mkdir /opt/oracle
cd /opt/oracle
unzip -q /tmp/Downloads/instantclient-basic-linux.x64-12.1.0.2.0.zip
unzip -q /tmp/Downloads/instantclient-sdk-linux.x64-12.1.0.2.0.zip
mv instantclient_12_1 instantclient
cd instantclient
ln -s libclntsh.so.12.1 libclntsh.so


Install Node.js


Use curl to download the Node.js software from nodejs.org and the linux pipe (|) function to pass it to the tar utility which unpacks the software in the /opt/ directory.
cd /opt
curl -sSL https://nodejs.org/dist/v4.4.2/node-v4.4.2-linux-x64.tar.xz | tar -xJC /opt/


Install node-oracledb module


The node-oracedb will be installed as global module by the npm (node package manager). Before running npm, set some environment parameters so the node binaries are in the search path and the Oracle libraries can be found.
export PATH=/opt/node-v4.4.2-linux-x64/bin:$PATH
export LD_LIBRARY_PATH=/opt/oracle/instantclient:$LD_LIBRARY_PATH
npm install -g oracledb


Create the jpoot/node_oracledb image.

exit
docker commit a42c4d9b4434 jpoot/node_oracledb:1.0
 

Testing the created Docker image


Now that I have created the Node.js enabled image, I can test the functionality of it.

I have downloaded examples scripts from de node-oracle/examples on Github.

Start with running a simple select1.js against an Oracle database. This script connects to the database and selects one row from the Departments table.
I have an Oracle Database 12.1.0.2.0 pluggable database running on a separate VM, with the Oracle example schema's installed in it.
I need to provide some environment variables to provide the PATH, user, password and connect string for the .js scripts to be able to connect to the database.

Create a file called env.list and place the following entries in it.
vi env.list
PATH=/opt/node-v4.4.2-linux-x64/bin:$PATH
LD_LIBRARY_PATH=/opt/oracle/instantclient:$LD_LIBRARY_PATH
NODE_PATH=/opt/node-v4.4.2-linux-x64/lib/node_modules
NODE_ORACLEDB_USER=hr
NODE_ORACLEDB_PASSWORD=hr
NODE_ORACLEDB_CONNECTIONSTRING=192.168.100.45:1521/fmwdb1.domain.local


Explanation of the environment variables

PATH - Add the path to the node and npm binaries to the search path
LD_LIBRARY_PATH - Provides the path to the Oracle libraries
NODE_PATH - Provides the path to the global modules of Node.js
NODE_ORACLE_* - Provides the user, password and connect string to the .js scripts. See dbconfig.js for details

I know, I know… Putting a plaintext password in a file is not secure. Keep in mind that this is for demonstration purposes only.
Don't do this in any non-demo environment!!!

Run the container with the necessary parameters.

docker run --rm -u nodejs -w /home/nodejs/examples --env-file ./env.list \
 --add-host=db01.domain.local:192.168.100.45 -v /tmp:/home/nodejs/examples \
 jpoot/node_oracledb:1.0 node select1.js

Let's walk through the parameters of the docker run command I used.

docker run
Main docker command to run the container

--rm
Remove the container after it has completed

-u nodejs
User to run within the container

-w /home/nodejs/examples
Working directory, the container starts in this directory

--env-file ./env.list
File that contains the environment variables to be provided to the container

--add-host=db01.domain.local:192.168.100.45
Add a host entry for the database server to the /etc/hosts file. This is used because I don't have a DNS server running.

-v /tmp:/home/nodejs/examples
Add a volume to the container, mapping a local directory to a directory in the running container.
In this case this mounts a directory with the .js scripts I want to run. So you don't have to add the scripts to the image, making it not dependent on script changes.

jpoot/node_oracledb:1.0
Docker image used as base for the container. In this case it is the image I created earlier. node 

select1.js
Command to run in the container. Run node command with the select1.js script.



YES!!! The scripts ran correctly in the container.



And now the easy way


In the previous chapters I have created a Docker images that can run Node.js with the capability to connect to an Oracle database.
As you have seen, this involves a lot of manual labor. Manual labor, means greater change of mistakes.

So let's automate the way to create this image.

In the next steps I will create and use a Dockerfile to automated the creation of the image.

A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build, users can create an automated build that executes several command-line instructions in succession.

See. Best practices for writing Dockerfiles and Dockerfile reference for more details on the Dockerfile

Dockerfile


Create an empty directory to place the Dockerfile in.
I will use the same Instantclient zip files as previously downloaded.

instantclient-basic-linux.x64-12.1.0.2.0.zip
instantclient-sdk-linux.x64-12.1.0.2.0.zip

Put these files in the same directory as the Dockerfile for our convenience. (Or create a symlink to the files)
mkdir docker-file
cd docker-file
vi Dockerfile
# Pull Oracle Linux 7 image from Docker hub
FROM oraclelinux

 
# Install OS packages
RUN yum -y install unzip libaio gcc-c++ tar make curl \
&& useradd nodejs -p '$6$salt$ZjJzVKp5xtoIl7cfXqZe0mQjWeOpsV2pMiIYpWzkR4ExCBpPdT3mi3eXtG1MSawJnZfXFjBcq0UUmenLq1Cj//'

 
# Add Node.js
RUN curl -sSL https://nodejs.org/dist/v4.4.2/node-v4.4.2-linux-x64.tar.xz \
| tar -xJC /opt/

ENV PATH /opt/node-v4.4.2-linux-x64/bin:$PATH

# Add Oracle Instantclient
ADD instantclient-basic-linux.x64-12.1.0.2.0.zip /tmp/
ADD instantclient-sdk-linux.x64-12.1.0.2.0.zip /tmp/


RUN unzip -q /tmp/instantclient-basic-linux.x64-12.1.0.2.0.zip -d /opt/oracle/ \
&& unzip -q /tmp/instantclient-sdk-linux.x64-12.1.0.2.0.zip -d /opt/oracle/ \
&& mv /opt/oracle/instantclient_12_1 /opt/oracle/instantclient \
&& ln -s /opt/oracle/instantclient/libclntsh.so.12.1 /opt/oracle/instantclient/libclntsh.so\
&& rm /tmp/instantclient-*


ENV LD_LIBRARY_PATH /opt/oracle/instantclient

# Install the node-oracledb module as global module to Node.js using npm
RUN npm install -g oracledb


ENV NODE_PATH /opt/node-v4.4.2-linux-x64/lib/node_modules

Build the Docker image


From the docker-file directory run the docker build command an tag the created images with a name and version.
docker build -t="jpoot/node_oracledb:1.1" .
Note. Mind the . at the end of the command. It says, build the image using the Dockerfile here.



Skipping the yum install lines ….



Within two minutes I have a fully functional Docker image ready. J
docker images


Now let's see if the image works.

Remove the PATH, LD_LIBRARY_PATH and NODE_PATH from the env.list.
These environment variables where already provided in the Dockerfile.
docker run --rm -u nodejs -w /home/nodejs/examples --env-file ./env.list \
 --add-host=db01.domain.local:192.168.100.45 -v /tmp:/home/nodejs/examples \
 jpoot/node_oracledb:1.1 node select1.js



Yeah, it works as expected!!!


To test the functionality I used Node.js to run a sql script. The same Docker image can be used to run other .js scripts. As another example, run a simple webserver.
docker run -d -u nodejs -w /home/nodejs/examples -p 80:3000/tcp -v /tmp:/home/nodejs/examples
 jpoot/node_oracledb:1.1 node http.js



Btw. Don't forget to kill this running container when you're done.
docker kill <container id>

Remarks


In this article I have shown you two ways to create a Docker image Node.js with a functional node-oracledb module.
There is now right or wrong way to create the image. It is however much easier to use the Dockerfile method. It is fast, easy and prevents human errors. Also, if you want to add modules or functionality to the image just add the commands to the Dockerfile and create a new image in a couple of minutes.

Sources and references


nodejs.org
node-oracledb on Github
Oracle Instant Client on OTN
First setup of a connection from Node.js to an Oracle Database
Running node-oracledb - the Oracle Database Driver for Node.js - in the Pre Built VM for Database Development

Sunday, April 3, 2016

First setup of a connection from Node.js to an Oracle Database

In this article I will demonstrate how to make a connection to a remote Oracle database from Node.js running on linux 7. We will be using the node-oracledb module to accomplish this. My colleague Lucas Jellema gave a great explanation about this module in his recent article Running node-oracledb - the Oracle Database Driver for Node.js - in the Pre Built VM for Database Development

As described in that article, the node-oracldb module is depending on the Oracle 11.2 or 12.1 client libraries. So you need to install a full Oracle client, local database or the Oracle Instant Client. I will be using the Oracle Instant Client since it is small and easy to install.

Why Linux 7?

As of Node.js 4 the compiler must support C++11.
This is not included in the default compiler on Linux 6. You can either install another compiler or use Linux 7.

Setup

In this article I will be using the following setup.

Oracle Linux 7 VM on VirtualBox
Node.js 4.4.2 (64-bits)
node-oracledb 1.8
Oracle Instantclient 12.1.0.2.0

OS prerequisites

unzip
libaio
gcc-c++

Use yum to install the OS prerequisites.
yum install unzip libaio gcc-c++



On another VM, I have an Oracle Database 12.1.0.2.0 pluggable database running.

Download the components

node-v4.4.2-linux-x64.tar.xz from nodejs.org
instantclient-basic-linux.x64-12.1.0.2.0.zip from Oracle OTN
instantclient-sdk-linux.x64-12.1.0.2.0.zip from Oracle OTN
node-oracledb will be installed via node package manager, npm

Put the files in the /tmp directory of the VM using any sftp tool you like.


We will remove them when we are done.

Installing components

Logon as root (or use sudo)

Install Node.js

cd /opt
tar -Jxf /tmp/node-v4.4.2-linux-x64.tar.xz

Install Oracle instant client

mkdir /opt/oracle
cd /opt/oracle
unzip -q /tmp/instantclient-basic-linux.x64-12.1.0.2.0.zip
unzip -q /tmp/instantclient-sdk-linux.x64-12.1.0.2.0.zip

Rename the directory so we don't have to tell the installer where to find the OCI libraries, etc...
If you install the Oracle Instant Client in another location, you will have to set two environment variables, OCI_LIB_DIR and OCI_INC_DIR before installing th oracledb module. See INSTALL.md on Github for more details about this.

mv instantclient_12_1 instantclient
cd instantclient
ln -s libclntsh.so.12.1 libclntsh.so




Remove files from /tmp
rm /tmp/instantclient-* node-v4.4.2-linux-x64.tar.xz

Install oracledb module

You can choose to install the module local to the user of global for the system.
If you install it local to the user, you don't need to be a privileged user. You can choose any user you need to run Node.js.
For this demonstration I created a user called nodejs and will install the module local to the user.

Logon as user nodejs

Set environment variables

export PATH=/opt/node-v4.4.2-linux-x64/bin:$PATH
export LD_LIBRARY_PATH=/opt/oracle/instantclient:$LD_LIBRARY_PATH

npm install oracledb



 The module is created in /home/nodejs/node_modules



Test the module

On Github there are several example scripts available for use with the node-oracledb module. See. node-oracledb examples
Download dbconfig.js and select1.js for a test of a db connection.
You can either change the dbconfig.js to match you db connection or set some environment variables.

export=NODE_ORACLEDB_USER=hr
export NODE_ORACLEDB_PASSWORD=hr
export NODE_ORACLEDB_CONNECTIONSTRING=db01.domain.local:1521/fmwdb1.domain.local

Run the select1.js

This will perform a simple query on the departments table from the HR sample schema.



Some remarks


Set the environment variables permanent.

You can set the environment variables permanently for either the specific user or system wide.
Place them in the .bash_profile of the user that will run Node.js or create a .sh file in /etc/profile.d so the environment variables are set at logon for every user.

Install oracledb module global and set additional environment variable.

npm install -g oracledb
Set the environment variable NODE_PATH so Node.js knows where to find the modules.
export NODE_PATH=/opt/node-v4.4.2-linux-x64/lib/node_modules

Sources and references


nodejs.org node-oracledb on Github 
Oracle Instant Client on OTN 
Running node-oracledb - the Oracle Database Driver for Node.js - in the Pre Built VM for Database Development