Use ephemeral ports for idp-fixture

[jaymode]

This change removes the use of hardcoded port values for the
idp-fixture in favor of the mapped ephemeral ports. This should prevent
failures due to port conflicts in CI.

[elasticmachine]

Pinging @elastic/es-core-infra

[jkakavas]

We should also change

ports:
      - "4443:4443"

to

ports:
      - "4443"

in x-pack/test/idp-fixture/docker-compose.yml so that we get a random port on localhost mapped to the 4443 of the container so that the IDP can be reached

[jkakavas]

Rename task to copyFiles or something generic like that ?

[alpar-t]

Thanks for noticing and fixing this @jaymode !
The code to look up the ports is not necessary TestFixturesPlugin already sets this up,
postProcessFixture will have extension properties with the ports, so you just need something like this: postProcessFixture.doLast { println ext."test.fixtures.shibboleth-idp.tcp.443" } Note that any testing task also gets passed a system property by the same name.

[jaymode]

thanks for that tip!

[jkakavas]

This will unfortunately not work as is. The port is part of the SingleSignOnService binding URL and that URL is set as the Destination element value in the SAML Authentication Response. So, even if we set this to a random port outside the container, the Shibboleth IDP will still be configured with 4443 internally so from its perspective, the SingleSignOnService HTTP-Redirect binding Location will still be

https://localhost:4443/idp/profile/SAML2/Redirect/SSO

and it will fail on an authentication request with a Destination set to

https://localhost:34567/idp/profile/SAML2/Redirect/SSO

Setting the Destination in the authn request is only mandatory when the the request is signed ( otherwise optional ) but even if we were to change this default behavior , the tests would work only for a subset of use cases and then again something else might break.
Shibboleth IDP is not easy to run behind a reverse proxy :/ , I think we'd have to use something like httpd or nginx to do proper request rewriting so that both our saml realm and the Shibboleth IDP can be happy.

Alternatively, maybe @atorok knows if this is possible, we might be able to get around this if we can replace the random ephemeral port in the mounted volumes config files, so that shibboleth / tomcat use randomized ports internally also

[jaymode]

The port is part of the SingleSignOnService binding URL and that URL is set as the Destination element value in the SAML Authentication Response.

The idp-metadata.xml gets re-written with the updated port, which is how it works. Maybe shibboleth handles this better now? Or is the test broken?

The push from yesterday was missing some of my changes since I committed in a sub directory instead (DOH!).

[jkakavas]

The idp-metadata.xml gets re-written with the updated port, which is how it works

This is the idp-metadata.xml file that we copy to outputDir from the idp/shibboleth-idp/metadata/idp-metadata.xml and it is the one the saml realm uses in its configuration (after we rewrite the port).

Shibboleth internally still knows of the old idp-metadata.xml (mapped from idp/shibboleth-idp/metadata/idp-metadata.xml ) and this one still points to 4443 for the SingleSignOn port. So I would expect it to deny the authn request with a

AML message intended destination endpoint  X did not match the recipient endpoint Y

[jaymode]

The reason that this works is due to the way that Shibboleth gets the URL for comparison. It uses HttpServletRequest#getRequestURL. The jetty implementation builds this URL and uses data provided in the HOST header provided by the client, which has the ephemeral port and not the port that shibboleth has in its configuration.

[alpar-t]

LGTM

[jkakavas]

LGTM