Shibboleth: update shibboleth2.xml entityID to end with "sp" rather than "shibboleth"

[pdurbin]

Per https://spaces.internet2.edu/display/InCFederation/Entity+IDs it has been recommended to use to have our Shibboleth entityID be in this form...

entityID="https://sp_name.example.edu/sp"

... rather than this one:

entityID="https://sp_name.example.edu/shibboleth"

We should update the example at https://github.com/IQSS/dataverse/blob/master/conf/vagrant/etc/shibboleth/shibboleth2.xml

[pdurbin]

We should update the example at https://github.com/IQSS/dataverse/blob/master/conf/vagrant/etc/shibboleth/shibboleth2.xml

More importantly, we've already updated the example at http://guides.dataverse.org/en/latest/installation/shibboleth.html to be entityID="https://shibtest.dataverse.org/sp".

However, all of our servers still have the old style ending in "shibboleth" rather than "sp". For example, https://dataverse-demo.iq.harvard.edu/Shibboleth.sso/Metadata shows entityID="https://dataverse-demo.iq.harvard.edu/shibboleth".

@scolapasta and @kcondon I'm thinking we could use this issue to work on making sure all of our servers use the new ending. Also, we should re-define which servers we want Shibboleth support on. Here's a potential list:

• https://dataverse.harvard.edu
• https://dataverse-demo.iq.harvard.edu
• dvn-build
• dataverse-internal?
• http://beta.dataverse.org ?
• https://apitest.dataverse.org ?
• cluster/staging?

In INC01226245 we're working on getting https://dataverse.harvard.edu registered with InCommon at https://incommon.org/federation/info/all-entity-categories.html#SPs but we could probably have a few testing or staging instances. In the past we've done "out of band" metadata exchanges for Harvard's Identity Provider (IdP) but since the Harvard IdP is part of InCommon we should probably treat it like all other IdPs in that federation.

[bencomp]

The Entity IDs doc also has a warning that says

Do NOT change your entity ID!

So in any case it's important to choose wisely before registering and using an ID. Add to #2884?

[pdurbin]

https://dataverse-demo.iq.harvard.edu was renamed to https://demo.dataverse.org in #2651.

To me this issue is about registering the following two servers at https://incommon.org/federation/info/all-entities.html#SPs

• https://dataverse.harvard.edu (production)
• https://demo.dataverse.org (QA/testing/staging/demo)

As of this writing https://dataverse.harvard.edu is using a non-recommended entityID (doesn't end in "sp") and https://demo.dataverse.org isn't configured for Shibboleth at all (after the rename). To make progress on this I could at least reconfigure demo with a recommended entityID (I may want to change the docs from shibtest to demo) and get it working again with http://www.testshib.org .

Incidentally, I noticed that many QA/testing/staging/demo Service Providers are registered with InCommon...

... and I think one server should be enough but I'll check with @kcondon if he'd like more from the list at #2104 (comment) registered with InCommon.

[pdurbin]

I checked in with @kcondon and he said that ideally the following servers would be registered with InCommon:

• https://dataverse.harvard.edu (production) - public
• https://demo.dataverse.org (demo of latest release) - public
• https://beta.dataverse.org (demo of pre-release software) - public
• build server 1 (QA) - private
• build server 2 (QA) - private

I'm adding public vs. private above because I think that all servers would necessarily become public as part of the InCommon registration process. This means that they would need to be given stable hostnames and have real SSL certs, etc.

I don't think we should bother registering https://apitest.dataverse.org and in general I think there's agreement that this should some day be deprecated in favor of https://demo.dataverse.org

Likewise, it's probably not worth registering http://phoenix.dataverse.org . It's actively used but we don't need to clutter up the InCommon list and there seem to be enough other hosts to test on. Same with cluster/staging which is used for load testing.

I'm planning on using https://shibtest.dataverse.org for testing in the near term but I don't think it needs to be registered with InCommon either. We can use beta or some other server.

[pdurbin]

I just changed the entityID on https://demo.dataverse.org to the one we want to use (entityID="https://demo.dataverse.org/sp"). I changed it in two places:

[root@dataverse-demo ~]# cd /etc/shibboleth
[root@dataverse-demo shibboleth]# diff shibboleth2.xml.2016-02-04.oldhostname shibboleth2.xml
16c16
<     <ApplicationDefaults entityID="https://dataverse-demo.iq.harvard.edu/shibboleth"
---
>     <ApplicationDefaults entityID="https://demo.dataverse.org/sp"
31c31
<             <SSO discoveryProtocol="SAMLDS" discoveryURL="https://dataverse-demo.iq.harvard.edu/loginpage.xhtml">SAML2 SAML1</SSO>
---
>             <SSO discoveryProtocol="SAMLDS" discoveryURL="https://demo.dataverse.org/loginpage.xhtml">SAML2 SAML1</SSO>
[root@dataverse-demo shibboleth]# 

Running service shibd restart was enough to download the new metadata from https://demo.dataverse.org/Shibboleth.sso/Metadata but insufficient for login to work properly. I had to run service httpd restart as well. Now it works again for TestShib users (I re-uploaded the metadata per http://guides.dataverse.org/en/4.2.3/installation/shibboleth.html#testing ) but not for Harvard or MIT users. They'll see errors like below until we either join InCommon or ask for another out-of-band metadata exchange:

2016-03-07 update: The TestShib metadata expired again, resulting in the usual "Something horrible happened... Error Message: SAML 2 SSO profile is not configured for relying party" error:

Rather than re-uploading our metadata I configured https://demo.dataverse.org for InCommon (as I mentioned at #2937) even though we haven't been registered with them.

[pdurbin]

As of yesterday (INC01606454) https://dataverse.harvard.edu/Shibboleth.sso/Metadata and https://dataverse.harvard.edu/Shibboleth.sso/DiscoFeed are back in action. I chatted with @bencomp about this at http://irclog.iq.harvard.edu/dataverse/2016-02-25

We still need to (via INC01226245):

• register https://dataverse.harvard.edu with InCommon
• register https://demo.dataverse.org with InCommon
• register https://beta.dataverse.org with InCommon (can start work on Shibboleth: Support "Federated Login Mode" (i.e. feed of Identity Providers from InCommon) #2937 prior to this)

Once these are registered with InCommon I'll consider this issue ready for QA.

[pdurbin]

A week ago the people working on INC01226245 replied and said they are actively trying to resolve it.

[pdurbin]

@kcondon and I discussed how #2937 is really a better issue to leave open for getting Harvard servers registered with InCommon in INC01226245 since it's about federated login so I'm passing this to QA to close. I changed the title of this issue to be about changing the entityID to end in "sp" rather than "shibboleth" which we have done on all of our servers and in the guides.