2.2. Linux Production with Docker

Introduction

A basic HADatAc installation involves the following steps:

• acquiring required software for installing/running the HADatAc Console

• downloading HADatAc code from this GitHub repository

• deploying HADatAc

Step 1 : Acquiring Required Software

Before anything, however, we need to verify that the host machine has a git client to retrieve the code. In the command line of your machine, verify that you have a git client installed in your machine:

git

If you do not have git, you need to install one: http://git-scm.com/downloads.

Install a Docker Community Edition that can be downloaded from https://docs.docker.com/install/. Docker is available for many operating systems as listed in the link above.

Go here to get started with Docker.

Step 2: Cloning HADatAc Code

Once your are in your home directory (~ is a shortcut for your home directory), clone HADatAc code by typing the following command:

  git clone https://github.com/paulopinheiro1234/hadatac.git

After cloning HADatAC, you should have a hadatac folder under your home directory. Go into the hadatac directory under your home directory by typing the following:

 cd ~/hadatac

Step 3: Changing configuration files

The various HADatAc configuration files need to be changed before you build it with Docker. After you build the hadatac service, though, you can edit the configuration files inside of the container and restart the service if you make a mistake or miss something.

The following configuration files in particular, need to be changed to suit your needs:

• conf/hadatac-docker.conf - The base_url should be changed to the domain name where HADatAc is being deployed. Also, the community settings (at the bottom of the file) should be adjusted to your needs. Particularly, the ont_prefix setting should be properly set.
• conf/autoccsv-docker.config - The template_file_name setting should be set to the template file you plan to use for CSV header mapping.
• conf/play-authenticate/smtp.conf - The SMTP setting should be set properly to allow for sending registration emails.

Read this section for more information about the configuration files and their settings.

Step 4: Deploying HADatAc

In HADatAc's home directory type the following:

 docker-compose build

Followed by the following command:

 docker-compose up -d 

Wait about a minute, and if you deployed it successfully, you can access http://(domain):9000/hadatac, where (domain) is the name of the domain where you deployed it. By default HADatAc is run on port 9000, so for testing purposes, you should make sure that this port is open to the public.

In order to access HADatAc from port 80 or 443 (http and https ports), you need to install apache2 or nginx as described in the next section.

Step 4: Setting up Apache2 / Nginx Proxy

4.1: Apache2

In the instruction below we use apache2 for proxying hadatac from localhost:9000 into (domain)/hadatac

Assuming that apache2 is installed, change to apache2 folder in /etc

 cd /etc/apache2

in the /sites-available folder, edit the conf file. Look for

<VirtualHost *:80>

Right before the closing tag </VirtualHost>, add the following content:

 ProxyRequests Off

 <Proxy *>
     Order deny,allow
     Allow from all
 </Proxy>

 ProxyPass /hadatac http://127.0.0.1:9000/hadatac
 ProxyPassReverse /hadatac http://127.0.0.1:9000/hadatac

Save the file.

Go to http://(domain)/hadatac, where (domain) is the name of your domain

4.2: Nginx

Install nginx before following the next steps. Once it is installed, do the following.

cd /etc/nginx/sites-enabled

Edit the enabled site file, probably called default.

Add the following location block under the server block.

# Pass requests for /hadatac
location /hadatac
{
    proxy_pass http://localhost:9000;
    proxy_set_header Host $host;
    proxy_set_header Referer $http_referer;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-NginX-Proxy false;
    proxy_set_header X-Forwarded-Proto $scheme;
}

Save the file and restart nginx. On Ubuntu systems, you can restart nginx with sudo service nginx restart.

Other Information

Stopping HADatAc

To stop all of the HADatAc services deployed with Docker i.e. Blazegraph, Solr and the Play application, type docker-compose stop.

To tear down the services e.g. to completely remove the containers, type docker-compose down. This will still keep in place the various Docker Volumes that were attached to the Docker Containers. The volumes include a volume containing the Solr data, the Blazegraph data, the HADatAc configuration directory, and the directory containing all of the CSV files.

By keeping these volumes in tact, you can re-build HADatAc without losing any of the data already ingested. However, if you wish to completely remove the HADatAc instance, you can type docker-compose down -v and all of the volumes will be destroyed as well.

Re-building HADatAc

If changes are made to HADatAc, you will want to re-compile the source code inside of the Docker Container. To do this, you simply run the following commands.

docker-compose down
docker-compose build
docker-compose up -d

This will remove the old HADatAc containers, re-build them and then bring them back up. No data or configuration file changes will be lost though, since those are stored in Docker Volumes as previously discussed.

Editing HADatAc Configuration Files in a Built/Running Container

The various configuration files for HADatAc are stored inside of a Docker Volume. This means that the conf directory is independent of the running HADatAc Docker container. So, if you tear down the containers with a down command, the conf directory will still be present as an independent Docker volume with any changes you may have made.

Then, when you bring the HADatAc Docker container back up, the previous conf Docker volume will be attached to the conf directory inside of the new container, with any changes you made still present.

When you are already running HADatAc, the conf Docker volume has already been created. So, in order to edit the various configuration files of an already built HADatAc Docker instance, you will need to first run HADatAc and access the Bash terminal.

docker-compose up -d
docker-compose exec hadatac bash

Once you are inside of the container and at the Bash terminal, type the following commands.

cd conf
[vi or nano] <config filename>
# make any changes, and then save the file

After you have edited the files, you will need to exit the Bash terminal and restart HADatAc.

# Inside of the HADatAc Bash terminal, type...
exit
# From your local machine now, type...
docker-compose restart

This will restart HADatAc causing it to use the new configuration files.

Useful Docker Commands

Viewing Log Files

To view the logs files of the HADatAc play app, you can type docker-compose logs hadatac. If you wish to attach to the logs and see log messages as they are produced, you can type docker-compose logs -f hadatac.

To view Solr or Blazegraph logs, you can type docker-compose logs solr and docker-compose logs blazegraph, respectively.

Accessing the Bash Terminal of the HADatAc Container

To access a Bash terminal of the Docker Container that is running the HADatAc Play application, you can type docker-compose exec hadatac bash (make sure it is running first with docker-compose up -d). This will bring you into a Bash terminal inside of the container in the directory where the application code is stored.