Shibboleth users who predate Shibboleth are assumed to be email-verified but aren't

[donsizemore]

This issue is related to #1401 and possibly supersedes #3300. Per Phil and Pete in IRC Shibboleth users are now assumed to have their e-mail addresses verified as trusted / provided by the IdP. For new users this happens. Existing users such as myself and our archivists remain unverified. I can sign in with my Shibboleth account to our test server running 4.11 and don't get an "emailconfirmed" timestamp:

dvntest=> select emailconfirmed from authenticateduser where email='dls@email.unc.edu';
 emailconfirmed 
----------------

(1 row)

I could start wedging in timestamps for institutional users but would love to find the logic snag.

[pameyer]

I think the logic snag is that when the authenticateduser.emailconfirmed column was added in 4.5.1, it wasn't accompanied by an update setting that column to account creation time for shibboleth users.

[donsizemore]

@pameyer seems better to make current Shib logins check for emailconfirmed and set if necessary — we have a bunch of users matching unc.edu who may or may not sign in going forward. My inner Security Chicken Little would prefer to leave them NULL.

[pameyer]

@donsizemore That seems reasonable to me; but means it would take code changes rather than database cleanup.

[pdurbin]

I'm pretty sure I'm affected by this personally in Harvard Dataverse (I had an old account in the DVN 3.x days and I'm using Shib to log in) and the user experience is pretty terrible. Here are some screenshots from 5.10:

I noticed "Not Verified" and clicked "Verify Email".

I get a "success, email sent" message

The email I received is nonsensical (no link, for example, no content, really)

Knowing what I know from hacking on pull request #8579 recently, I'm aware that the user experience is even further confusing because prior to that pull request if you go back to Account Information and click "Verify Email" a pop up will show (screenshots below). What this popup is TRYING to say is that an email has already been sent. In the that pull request we decided to remove the popup. Here's how it looks in 5.10:

I'm back on "Account Information" (within 24 hours so the link/token is still present in the database)

Confusing popup that's trying to say an email has already been sent

Given all this, what's the plan? Some thoughts:

• We want Shib users to always see "Verified". They should never see a "Send Verification Email" button. They should never have the user experience above.
• From a quick look at the code, emailLastConfirmed (a timestamp) is only set for Shib users on account creation. Perhaps this should be set every time the Shib user logs in. That way, both new users and returning users would be treated the same. (Hopefully installations aren't relying on emailLastConfirmed not changing.) As users log in, emailLastConfirmed will be populated for the ones who didn't have it. That means they won't see the "Send Verification Email" button.

---

Update: Examples of the mostly empty nonsensical email above:

• https://help.hmdc.harvard.edu/Ticket/Display.html?id=316215
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=315336
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=312844
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=312844
• https://help.hmdc.harvard.edu/Ticket/Display.html?id=312043

[donsizemore]

@pdurbin I love the idea of populating emailLastConfirmed on each login just as Shibboleth attributes would update stored database fields. Q: how/will this affect OAuth users? (thinking OIDC)

[pdurbin]

@donsizemore great. Update at each login. Sounds like a plan.

OAuth and OIDC users would not be affected by any change I'm planning for this issue. For OIDC, please note that there's an open issue about email verification:

• OIDC: make use of attribute "email_verified" #6679

Meanwhile I did some more Shib testing locally and wanted to share some screenshots and a bug I'd like to fix.

When you create a Shib account, the process is fine (email shows as verified) except that the in-app notification says, "check for your welcome email to verify your address." This part of the message shouldn't be shown to Shib users. The email itself is fine. That is, the email correctly does not include a link or any text about verifying your email.

---

Subject: Root: Your account has been created

Hello,
Welcome to Root! Get started by adding or finding data. Have questions? Check out the User Guide at https://guides.dataverse.org/en/5.10/user or contact Root Support Team at support@mailinator.com for assistance.

You may contact us for support at support@mailinator.com.

Thank you,
Root Support Team

---

I have the change queued up locally to update the in-app notification for newly created Shib users.

[pdurbin]

More screenshots. I also looked at converting accounts from builtin to Shib. The bug here is that the email is not automatically set to verified.

Another potential bug is that a "confirm email" token is in the database for this user. Presumably this token was created back when I created the builtin user. Perhaps the conversion process to Shib should clear out any "confirm email" tokens. In practice, this is not a big problem because this token will simply expire in 24 hours (by default).

[pdurbin]

I put fixes in this pull request:

• improve email verification, no popup, auto-verify Shib users #8579