Skip to content

Commit

Permalink
Documentation changes reflecting the 4.3 installer fixes and improvem…
Browse files Browse the repository at this point in the history
…ents.
  • Loading branch information
landreev committed Mar 11, 2016
1 parent 7e9f2d9 commit a83066f
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 19 deletions.
24 changes: 20 additions & 4 deletions doc/sphinx-guides/source/installation/installation-main.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ Running the Dataverse Installer
-------------------------------

A scripted, interactive installer is provided. This script will configure your Glassfish environment, create the database, set some required options and start the application. Some configuration tasks will still be required after you run the installer! So make sure to consult the next section.
At this point the installer only runs on RHEL 6 and similar.
At this point the installer only runs on RHEL 6 and similar and MacOS X (recommended as the platform for developers).

Generally, the installer has a better chance of succeeding if you run it against a freshly installed Glassfish node that still has all the default configuration settings. In any event, please make sure that it is still configured to accept http connections on port 8080 - because that's where the installer expects to find the application once it's deployed.


You should have already downloaded the installer from https://github.com/IQSS/dataverse/releases when setting up and starting Solr under the :doc:`prerequisites` section. Again, it's a zip file with "dvinstall" in the name.

Expand All @@ -21,7 +24,12 @@ Execute the installer script like this::
# cd dvinstall
# ./install

The script will prompt you for some configuration values. If this is a test/evaluation installation, it should be safe to accept the defaults for most of the settings:
**NEW in Dataverse 4.3:** It is no longer necessary to run the installer as root!
Just make sure the user that runs the installer has the write permission in the Glassfish directory. For example, if your Glassfish directory is owned by root, and you try to run the installer as a regular user, it's not going to work.
(Do note, that you want the Glassfish directory to be owned by the same user that will be running Glassfish. And you most likely won't need to run it as root. The only reason to run it as root would be to be able to run the application on the default HTTP(S) ports 80 and 443, instead of 8080 and 8181. However, an easier, and more secure way to achieve that would be to instead keep Glassfish running on a high port, and hide it behind an Apache Proxy, via AJP, running on port 80. This configuration is in fact required if you choose to have your Dataverse support Shibboleth authentication. See more discussion on this here: :doc:`shibboleth`.)


The script will prompt you for some configuration values. If this is a test/evaluation installation, it may be possible to accept the default values provided for most of the settings:

- Internet Address of your host: localhost
- Glassfish Directory: /usr/local/glassfish4
Expand All @@ -36,16 +44,24 @@ The script will prompt you for some configuration values. If this is a test/eval
- Rserve User Name: rserve
- Rserve User Password: rserve

**New, as of 4.3:**

- Administration Email address for the installation;
- Postgres admin password - We'll need it in order to create the database and user for the Dataverse to use, without having to run the installer as root. If you don't know your Postgres admin password, you may simply set the authorization level for localhost to "trust" in the PostgresQL ``pg_hba.conf`` file (See the PostgresQL section in the Prerequisites). If this is a production evnironment, you may want to change it back to something more secure, such as "password" or "md5", after the installation is complete.
- Network address of a remote SOLR search engine service (if needed) - In most cases, you will be running your SOLR server on the same host as the Dataverse application (then you will want to leave this set to the default value of ``LOCAL``). But in a serious production environment you may set it up on a dedicated separate server.
- The URL of the TwoRavens application GUI, if this Dataverse node will be using a companion TwoRavens installation. Otherwise, leave it set to ``NOT INSTALLED``.


The script is to a large degree a derivative of the old installer from DVN 3.x. It is written in Perl. If someone in the community is eager to rewrite it, perhaps in a different language, please get in touch. :)

All the Glassfish configuration tasks performed by the installer are isolated in the shell script ``dvinstall/glassfish-setup.sh`` (as ``asadmin`` commands).

As the installer finishes, it mentions a script called ``post-install-api-block.sh`` which is **very important** to execute for any production installation of Dataverse. Security will be covered in :doc:`config` section but for now, let's make sure your installation is working.
Please note, that "out of the box" the installer will configure the Dataverse to leave unrestricted access to the administration APIs from (and only from) localhost. As the installer finishes, it mentions a script called ``post-install-api-block.sh`` which you may choose to execute in order to further restrict access to the administration APIs. This can be of special importance in a production installation of Dataverse. Security will be covered in :doc:`config` section but for now, let's make sure your installation is working.

This comment has been minimized.

Copy link
@pdurbin

pdurbin Mar 11, 2016

Member

@landreev I see you softened my warning about the importance of running the post-install-api-block.sh script. I'd go the other way and put VERY IMPORTANT in all caps. :)

On a related note, the installer ends with a fairly bland statement that we should probably improve to emphasize the security implications of only considering the running of the script and not actually running it: "Setup done. Consider running post-install-api-block.sh for blocking the sensitive API."

See also:

This comment has been minimized.

Copy link
@landreev

landreev via email Mar 11, 2016

Author Contributor

This comment has been minimized.

Copy link
@pdurbin

pdurbin Mar 11, 2016

Member

@landreev I think the only thing you're missing is that @michbarsinai is the one who worked on all the API blocking code, not me :)

So it's not me making it an optional step. Maybe we should change the blocking script to run automatically... the only problem is that I rely on the API endpoints being open in my REST Assured tests at https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-apitest-develop/ ... maybe setup-all could accept a flag for --insecure and I'd use it on that http://phoenix.dataverse.org server... this is all #1886 territory.

Oh, speaking of docs, since you're hacking on the Installation Guide anyway, you could try to knock out a few of the dozen or so open issues at #2944.

This comment has been minimized.

Copy link
@landreev

landreev via email Mar 11, 2016

Author Contributor

This comment has been minimized.

Copy link
@pdurbin

pdurbin Mar 11, 2016

Member

Yeah, should we create a new issue about --insecure mode? (Or put #1886 through QA?) I ask because I want to add --insecure to the call to setup-all.sh in this file: https://github.com/IQSS/dataverse/blob/4.3-installer-improvements/scripts/deploy/phoenix.dataverse.org/post

You're welcome to roll this into your change in your branch. Or ping me and I can add it. That phoenix post script gets called from https://build.hmdc.harvard.edu:8443/job/phoenix.dataverse.org-deploy-develop/

This comment has been minimized.

Copy link
@landreev

landreev via email Mar 11, 2016

Author Contributor

Logging In
----------

Out of the box, Glassfish runs on port 8080 and 8181 rather than 80 and 443, respectively, so visiting http://localhost:8080 (substituting your hostname) should bring up a login page. See the :doc:`shibboleth` page for more on ports, but for now, let's confirm we can log in by using port 8080. Poke a temporary hole in your firewall.
Out of the box, Glassfish runs on port 8080 and 8181 rather than 80 and 443, respectively, so visiting http://localhost:8080 (substituting your hostname) should bring up a login page. See the :doc:`shibboleth` page for more on ports, but for now, let's confirm we can log in by using port 8080. Poke a temporary hole in your firewall, if needed.

Superuser Account
+++++++++++++++++
Expand Down
24 changes: 9 additions & 15 deletions doc/sphinx-guides/source/installation/prerequisites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,29 +91,23 @@ The version that ships with RHEL 6 and above is fine::
# service postgresql initdb
# service postgresql start

Configure Access to PostgreSQL for the Installer Script
=======================================================
Configure Database Access for the Dataverse Application (and the Dataverse Installer)
=====================================================================================

- When using localhost for the database server, the installer script needs to have direct access to the local PostgreSQL server via Unix domain sockets. This is configured by the line that starts with ``local all all`` in the pg_hba.conf file. The location of this file may vary depending on the distribution. But if you followed the suggested installation instructions above, it will be ``/var/lib/pgsql/data/pg_hba.conf`` on RHEL and similar. Make sure the line looks like this (it will likely be pre-configured like this already)::

local all all peer

- If the installer still fails to connect to the databse, we recommend changing this configuration entry to ``trust``::

local all all trust
- The application will be talking to PostgreSQL over TCP/IP, using password authentication. If you are running PostgreSQL on the same server as Glassfish, we strongly recommend that you use the localhost interface to connect to the database. Make you sure you accept the default value ``localhost`` when the installer asks you for the PostgreSQL server address. Then find the localhost (127.0.0.1) entry that's already in the ``pg_hba.conf`` and modify it to look like this::

This is a security risk, as it opens your database to anyone with a shell on your server. It does not however compromise remote access to your system. Plus you only need this configuration in place to run the installer. After it's done, you can safely reset it to how it was configured before.
host all all 127.0.0.1/32 md5

- The Dataverse installer script (for more information on the installer, please see :doc:`installation-main`) will need to connect to PostgreSQL as the admin user, in order to create and set up the database that the Dataverse will be using. If for whatever reason it is failing to connect (for example, if you don't remember what your Postgres admin password is), you may choose to temporarily disable all the access restrictions on localhost connections, by changing the above line to::

Configure Database Access for the Dataverse Application
=======================================================
host all all 127.0.0.1/32 trust

- The application will be talking to PostgreSQL over TCP/IP, using password authentication. If you are running PostgreSQL on the same server as Glassfish, we strongly recommend that you use the localhost interface to connect to the database. Make you sure you accept the default value ``localhost`` when the installer asks you for the PostgreSQL server address. Then find the localhost (127.0.0.1) entry that's already in the ``pg_hba.conf`` and modify it to look like this::
Note that this rule opens access to the database server **via localhost only**. Still, in a production environment, this may constitute a security risk. So you will likely want to change it back to "md5" once the installer has finished.

host all all 127.0.0.1/32 password

- If the Dataverse application is running on a different server, you will need to add a new entry to the ``pg_hba.conf`` granting it access by its network address::

host all all [ADDRESS] 255.255.255.255 password
host all all [ADDRESS] 255.255.255.255 md5

(``[ADDRESS]`` should be the numeric IP address of the Glassfish server).

Expand Down

2 comments on commit a83066f

@landreev
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, here's what the paragraph in the guides is saying now:

IMPORTANT: Please note, that "out of the box" the installer will configure the Dataverse to leave unrestricted access to the administration APIs from (and only from) localhost. Please consider the security implications of this arrangement (anyone with shell access to the server can potentially mess with your Dataverse). An alternative solution would be to block open access to these sensitive API endpoints completely; and to only allow requests supplying a pre-defined "unblocking token" (password). If you prefer that as a solution, please consult the supplied script post-install-api-block.sh for examples on how to set it up.

And this is what we have in the post-install-api-block scirpt now:

# This leaves /api/admin and /api/test blocked to all connections except from those
# coming from localhost:
curl -X PUT -d localhost-only http://localhost:8080/api/admin/settings/:BlockedApiPolicy
curl -X PUT -d admin,test http://localhost:8080/api/admin/settings/:BlockedApiEndpoints

# In some situations, you may prefer an alternative solution - to block ALL connections to 
# these endpoints completely; but allow connections authenticated with the defined
# "unblock key" (password):
#curl -X PUT -d YOURSUPERSECRETUNBLOCKKEY http://localhost:8080/api/admin/settings/:BlockedApiKey
#curl -X PUT -d localhost-only http://localhost:8080/api/admin/settings/:BlockedApiPolicy

@landreev
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(ok, the formatting above is github interpreting shell comments as headings)

Please sign in to comment.