[Serverless] Allow authentication via the Elasticsearch JWT realm with the shared_secret client authentication type.

[azasypkin]

Summary

In this pull request, we are removing a temporary workaround that enabled the collection of extended Kibana metrics anonymously. Now, the agents authenticate to Kibana via JWT in order to collect the metrics. To support this type of client credentials, Kibana needs to forward the Es-Client-Authentication HTTP header with a shared secret to Elasticsearch during authentication. By default, Kibana forwards only Authorization header by default.

NOTE: Right now Kibana doesn't need to support JWT authentication outside of Serverless, but eventually we might extend the default value of elasticsearch.requestHeadersWhitelist to include Es-Client-Authentication.

Testing

Generate keys

NOTE: Test keys mentioned here aren't secret and already publicly exposed.

openssl genrsa 2048 > jwks_private.pem
openssl rsa -in jwks_private.pem -pubout > jwks_public.pem

## https://github.com/dannycoates/pem-jwk
pem-jwk jwks_public.pem > jwks.json

## Edit generated `jwks.json` to follow the format shown below:
## { key } -> { keys: [{ key }] }

JWKS (for xpack.security.authc.realms.jwt.jwt1.pkc_jwkset_path)

{
  "keys": [
    {
      "kty": "RSA",
      "e": "AQAB",
      "use": "sig",
      "n": "v9-88aGdE4E85PuEycxTA6LkM3TBvNScoeP6A-dd0Myo6-LfBlp1r7BPBWmvi_SC6Zam3U1LE3AekDMwqJg304my0pvh8wOwlmRpgKXDXjvj4s59vdeVNhCB9doIthUABd310o9lyb55fWc_qQYE2LK9AyEjicJswafguH6txV4IwSl13ieZAxni0Ca4CwdzXO1Oi34XjHF8F5x_0puTaQzHn5bPG4fiIJN-pwie0Ba4VEDPO5ca4lLXWVi1bn8xMDTAULrBAXJwDaDdS05KMbc4sPlyQPhtY1gcYvUbozUPYxSWwA7fZgFzV_h-uy_oXf1EXttOxSgog1z3cJzf6Q"
    }
  ]
}

Public key

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv9+88aGdE4E85PuEycxT
A6LkM3TBvNScoeP6A+dd0Myo6+LfBlp1r7BPBWmvi/SC6Zam3U1LE3AekDMwqJg3
04my0pvh8wOwlmRpgKXDXjvj4s59vdeVNhCB9doIthUABd310o9lyb55fWc/qQYE
2LK9AyEjicJswafguH6txV4IwSl13ieZAxni0Ca4CwdzXO1Oi34XjHF8F5x/0puT
aQzHn5bPG4fiIJN+pwie0Ba4VEDPO5ca4lLXWVi1bn8xMDTAULrBAXJwDaDdS05K
Mbc4sPlyQPhtY1gcYvUbozUPYxSWwA7fZgFzV/h+uy/oXf1EXttOxSgog1z3cJzf
6QIDAQAB
-----END PUBLIC KEY-----

Private key

-----BEGIN PRIVATE KEY-----
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC/37zxoZ0TgTzk
+4TJzFMDouQzdMG81Jyh4/oD513QzKjr4t8GWnWvsE8Faa+L9ILplqbdTUsTcB6Q
MzComDfTibLSm+HzA7CWZGmApcNeO+Pizn2915U2EIH12gi2FQAF3fXSj2XJvnl9
Zz+pBgTYsr0DISOJwmzBp+C4fq3FXgjBKXXeJ5kDGeLQJrgLB3Nc7U6LfheMcXwX
nH/Sm5NpDMefls8bh+Igk36nCJ7QFrhUQM87lxriUtdZWLVufzEwNMBQusEBcnAN
oN1LTkoxtziw+XJA+G1jWBxi9RujNQ9jFJbADt9mAXNX+H67L+hd/URe207FKCiD
XPdwnN/pAgMBAAECggEADiKRbMuXIsS2k7fjxGoFA5OQdCn5y8tt7o847+ivhJ5P
I3GHNJSdbt/yMlfi0tCkhEjQ6iSzjy8HUWA0CXeNRUwznEhXkOuIqsui6hNMHTkU
RLUplj63g1AcAtyZH7DUW5pKbcSanw4lLRPaIL2MxdoFCqH6WD+2e12+tFjAvHVc
Bm03+hIt2898ruLQfHLQ1MUegTmXWZ9fqiPizuKwrW9xlCGQbIKoqVHebCMqRFK0
XGv0NNmUTnNo+uF3++yYHv/TL96EFTmU7QhUFrddONzUXDv0JjyiOnHk32191R9m
V8Y9mq+RT+tMsu+dxr2Yk4Qc5oHYX84p/afQxX8sMQKBgQD9FUsnJASMeg4jNlq8
XjDsXWu0NoVGTfU9z/9/SU+L6KexubdCoCWxs+8KyA69PZkMW6HMw6BGuDPjTvI9
1DmRdnVEa5EmUv/CIXZcAM+9Q7yWEocB/JeQOj9mdC/u1/sdQxNg1ae2HqJczl2I
EO4r7YshHQmqCju4lfyEf4rWTwKBgQDCFdm535BIPa4wGRTD7tY/VOCpDeom+PxH
9LUhefJV+G9RhP2jEPW+9D/ux8YAkL4c2kZR9kddLVFAD8jwormjWk6+uL5/jAAI
j6Aor3spBTNpgji6YnRaIk2PDIznFnSDPhdoWGsw8QQQ54wUO8m51cqPSYVZIu2d
U0yYacGQRwKBgFuJkB0gEeUdYG+sATWQe/GB+Kq97YZ4O/OXf7nyMitQgxbtLTOT
6Q5VHmiv42TfGrQ1kFgXiakKhvn4W/WxBQFv7wpIPb+21XrJz52HTZwPG+7L1LkL
O2aXKsdLzup8g/8Ze7DSlk5w1hjrKzlDpmGNEX1wm0Y9XUxuM19ZIkZRAoGAFR/F
s8pWbNZxyABi1zR+kyQM07mU+6rr4nUK5drc+mhwzUGZTY9CAAebkcSik1stpfxH
3RHeEJEnH77YEwDTDal9mpqG+WDmfAgN2X/H+t37C4fF3ttqaIkFQgWOrHQwODyg
1ZWSDSCeXayl/WnIefZ/9np9DgeULyRq2Mfh7m8CgYEAsEhsZyAe7QrCoVMykUSp
sys7qht/9B4QgaeX1pPdaJxLTPIKG0gYldWF1/zyMtiCYVY+MJkwgaVRgjX+8Swa
QfluMQ4YYrurxcdn+nSggGGinM6rt0C319sduKouKChMpzoEQp2RAIH35Of2usmR
k8z9rWWE/VEBo6K1d+3g/rA=
-----END PRIVATE KEY-----

Generate JWT

• Go to https://jwt.io/
• Use the following header

{
  "alg": "RS256",
  "typ": "JWT"
}

• Use the following payload

{
  "iss": "https://kibana.elastic.co/jwt/",
  "sub": "aleh.zasypkin@elastic.co",
  "aud": "elasticsearch",
  "name": "Aleh Zasypkin",
  "iat": 946684800,
  "exp": 4070908800
}

• For signature use the private and public keys defined in the previous section
• Use generated token for authentication, e.g. eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2tpYmFuYS5lbGFzdGljLmNvL2p3dC8iLCJzdWIiOiJhbGVoLnphc3lwa2luQGVsYXN0aWMuY28iLCJhdWQiOiJlbGFzdGljc2VhcmNoIiwibmFtZSI6IkFsZWggWmFzeXBraW4iLCJpYXQiOjk0NjY4NDgwMCwiZXhwIjo0MDcwOTA4ODAwfQ.LBwLDK4CCYHjtmWZ_J0IwKP6BQjH-8LbKUu1Obj2bUAtZcGVrnO_pY1JXCG582BLegq8_RrlxZ0C8GKN-kvuFt7okPEkMqfT6yCi_gt271Xzlbe01IT6DX5WRm7nT6mjNI4USndemquxl0NxHCm07azKD4MUsYIlgp_YW14ZKmHn4fJW0qgDgt4CeRkLQm5QE--rZ7VnlOFvaAsIlC7bLHHvhj_ntMSraFJEXc1JE7va8QX_D6cXpHbszGjnm9G928gJ24XVjUqXuR23yDNcc6socTPbq8WO9tj67cknCZG1An1wtefDOOKiqMKhrHPvBz9eT1CnOm57l63K8LvulQ

Run ES

$ yarn es snapshot --license trial \
    -E xpack.security.authc.token.enabled=true \
    -E xpack.security.authc.realms.native.native1.order=0 \
    -E xpack.security.authc.realms.jwt.jwt1.order=1 \
    -E xpack.security.authc.realms.jwt.jwt1.token_type=access_token \
    -E xpack.security.authc.realms.jwt.jwt1.client_authentication.type=shared_secret \
    -E xpack.security.authc.realms.jwt.jwt1.client_authentication.shared_secret=my_super_secret \
    -E xpack.security.authc.realms.jwt.jwt1.allowed_issuer=https://kibana.elastic.co/jwt/ \
    -E xpack.security.authc.realms.jwt.jwt1.allowed_subjects=aleh.zasypkin@elastic.co \
    -E xpack.security.authc.realms.jwt.jwt1.allowed_audiences=elasticsearch \
    -E xpack.security.authc.realms.jwt.jwt1.allowed_signature_algorithms=[RS256] \
    -E xpack.security.authc.realms.jwt.jwt1.claims.principal=sub \
    -E xpack.security.authc.realms.jwt.jwt1.pkc_jwkset_path=/.../jwks.json <--- CHANGE PATH

Run Kibana

yarn start --serverless

Authenticate with JWT

Elasticsearch endpoint

GET http://localhost:9200/_security/_authenticate
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2tpYmFuYS5lbGFzdGljLmNvL2p3dC8iLCJzdWIiOiJhbGVoLnphc3lwa2luQGVsYXN0aWMuY28iLCJhdWQiOiJlbGFzdGljc2VhcmNoIiwibmFtZSI6IkFsZWggWmFzeXBraW4iLCJpYXQiOjk0NjY4NDgwMCwiZXhwIjo0MDcwOTA4ODAwfQ.LBwLDK4CCYHjtmWZ_J0IwKP6BQjH-8LbKUu1Obj2bUAtZcGVrnO_pY1JXCG582BLegq8_RrlxZ0C8GKN-kvuFt7okPEkMqfT6yCi_gt271Xzlbe01IT6DX5WRm7nT6mjNI4USndemquxl0NxHCm07azKD4MUsYIlgp_YW14ZKmHn4fJW0qgDgt4CeRkLQm5QE--rZ7VnlOFvaAsIlC7bLHHvhj_ntMSraFJEXc1JE7va8QX_D6cXpHbszGjnm9G928gJ24XVjUqXuR23yDNcc6socTPbq8WO9tj67cknCZG1An1wtefDOOKiqMKhrHPvBz9eT1CnOm57l63K8LvulQ
ES-Client-Authentication: SharedSecret my_super_secret
Accept: application/json

{
  "username": "aleh.zasypkin@elastic.co",
  "roles": [],
  "full_name": null,
  "email": null,
  "metadata": {
    "jwt_claim_aud": [
      "elasticsearch"
    ],
    "jwt_claim_name": "Aleh Zasypkin",
    "jwt_claim_iss": "https://kibana.elastic.co/jwt/",
    "jwt_token_type": "access_token",
    "jwt_claim_sub": "aleh.zasypkin@elastic.co"
  },
  "enabled": true,
  "authentication_realm": {
    "name": "jwt1",
    "type": "jwt"
  },
  "lookup_realm": {
    "name": "jwt1",
    "type": "jwt"
  },
  "authentication_type": "realm"
}

Kibana endpoint

GET http://localhost:5601/internal/security/me
Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJodHRwczovL2tpYmFuYS5lbGFzdGljLmNvL2p3dC8iLCJzdWIiOiJhbGVoLnphc3lwa2luQGVsYXN0aWMuY28iLCJhdWQiOiJlbGFzdGljc2VhcmNoIiwibmFtZSI6IkFsZWggWmFzeXBraW4iLCJpYXQiOjk0NjY4NDgwMCwiZXhwIjo0MDcwOTA4ODAwfQ.LBwLDK4CCYHjtmWZ_J0IwKP6BQjH-8LbKUu1Obj2bUAtZcGVrnO_pY1JXCG582BLegq8_RrlxZ0C8GKN-kvuFt7okPEkMqfT6yCi_gt271Xzlbe01IT6DX5WRm7nT6mjNI4USndemquxl0NxHCm07azKD4MUsYIlgp_YW14ZKmHn4fJW0qgDgt4CeRkLQm5QE--rZ7VnlOFvaAsIlC7bLHHvhj_ntMSraFJEXc1JE7va8QX_D6cXpHbszGjnm9G928gJ24XVjUqXuR23yDNcc6socTPbq8WO9tj67cknCZG1An1wtefDOOKiqMKhrHPvBz9eT1CnOm57l63K8LvulQ
ES-Client-Authentication: SharedSecret my_super_secret
Accept: application/json

{
  "username": "aleh.zasypkin@elastic.co",
  "roles": [],
  "full_name": null,
  "email": null,
  "metadata": {
    "jwt_token_type": "access_token",
    "jwt_claim_iss": "https://kibana.elastic.co/jwt/",
    "jwt_claim_name": "Aleh Zasypkin",
    "jwt_claim_aud": [
      "elasticsearch"
    ],
    "jwt_claim_sub": "aleh.zasypkin@elastic.co"
  },
  "enabled": true,
  "authentication_realm": {
    "name": "jwt1",
    "type": "jwt"
  },
  "lookup_realm": {
    "name": "jwt1",
    "type": "jwt"
  },
  "authentication_type": "realm",
  "authentication_provider": {
    "type": "http",
    "name": "__http__"
  },
  "elastic_cloud_user": false
}

cc @lukeelmers @elastic/kibana-security

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 0df7659

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
enterpriseSearch	14	16	+2
securitySolution	408	412	+4
total			+6

Total ESLint disabled count

id	before	after	diff
enterpriseSearch	15	17	+2
securitySolution	487	491	+4
total			+6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[ymao1]

LGTM