Support SPNEGO/Kerberos auth to Elasticsearch

[jbaiera]

Work is wrapping up on adding a Kerberos based authentication realm to Elasticsearch. The realm will secure the rest endpoints by means of SPNEGO. Since Kerberos is used heavily in the Big Data space, we should offer an integration that allows for ES-Hadoop to use the currently logged in user and their Kerberos Ticket to authenticate to Elasticsearch over SPNEGO. This should include token authentication and appropriate refreshing mechanisms to avoid the same pitfalls that Hadoop runs into with KDC spamming and token lifetime.

[vamsi360]

@jbaiera
This is great to know. We can now integrate ES also into a centralised Kerberos cluster. Any estimate of when this is going to be released?

[jbaiera]

There is a substantial amount of work that is already completed for this feature, but there is quite a bit more to go, including the extensive testing that we will need to eventually run to ensure correctness.

Additionally, we are working out the best course of action for handling token authentication in worker tasks. The current token functionality in ES is based on OAuth username and password grants, which does not provide tokens that are useable in Hadoop/Spark/Storm (the tokens change between refreshes, last only an hour, can only be refreshed once, etc).

Hopefully soon I will be able to pull together a preliminary PR for the work, but until then, I have the work periodically checked into a feature branch located at master...jbaiera:feature-kerberos

[jbaiera]

This is currently waiting on #1183

[jbaiera]

Closed in error.

[jbaiera]

This is currently waiting on elastic/elasticsearch#34383 for further development

[jbaiera]

#1244 is merged now.