[Rollup] Add new capabilities endpoint for concrete rollup indices (#…

…32111)

This introduces a new GetRollupIndexCaps API which allows the user to retrieve rollup capabilities of a specific rollup index (or index pattern). This is distinct from the existing RollupCaps endpoint.

- Multiple jobs can be stored in multiple indices and point to a single target data index pattern (logstash-*). The existing API finds capabilities/config of all jobs matching that data index pattern.
- One rollup index can hold data from multiple jobs, targeting multiple data index patterns. This new API finds the capabilities based on the concrete rollup indices.

x-pack/docs/en/rest-api/rollup/rollup-index-caps.asciidoc

@@ -0,0 +1,161 @@
+[role="xpack"]
+[[rollup-get-rollup-index-caps]]
+=== Get Rollup Index Capabilities
+++++
+<titleabbrev>Get Rollup Index Caps</titleabbrev>
+++++
+
+experimental[]
+
+This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored).
+A single rollup index may store the data for multiple rollup jobs, and may have a variety of capabilities depending on those jobs.
+
+This API will allow you to determine:
+
+1. What jobs are stored in an index (or indices specified via a pattern)?
+2. What target indices were rolled up, what fields were used in those rollups and what aggregations can be performed on each job?
+
+==== Request
+
+`GET {index}/_xpack/rollup/data`
+
+//===== Description
+
+==== Path Parameters
+
+`index`::
+  (string) Index or index-pattern of concrete rollup indices to check for capabilities.
+
+
+
+==== Request Body
+
+There is no request body for the Get Jobs API.
+
+==== Authorization
+
+You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API.
+For more information, see
+{xpack-ref}/security-privileges.html[Security Privileges].
+
+==== Examples
+
+Imagine we have an index named `sensor-1` full of raw data.  We know that the data will grow over time, so there
+will be a `sensor-2`, `sensor-3`, etc.  Let's create a Rollup job, which stores it's data in `sensor_rollup`:
+
+[source,js]
+--------------------------------------------------
+PUT _xpack/rollup/job/sensor
+{
+    "index_pattern": "sensor-*",
+    "rollup_index": "sensor_rollup",
+    "cron": "*/30 * * * * ?",
+    "page_size" :1000,
+    "groups" : {
+      "date_histogram": {
+        "field": "timestamp",
+        "interval": "1h",
+        "delay": "7d"
+      },
+      "terms": {
+        "fields": ["node"]
+      }
+    },
+    "metrics": [
+        {
+            "field": "temperature",
+            "metrics": ["min", "max", "sum"]
+        },
+        {
+            "field": "voltage",
+            "metrics": ["avg"]
+        }
+    ]
+}
+--------------------------------------------------
+// CONSOLE
+// TEST[setup:sensor_index]
+
+If at a later date, we'd like to determine what jobs and capabilities were stored in the `sensor_rollup` index, we can use the Get Rollup
+Index API:
+
+[source,js]
+--------------------------------------------------
+GET /sensor_rollup/_xpack/rollup/data
+--------------------------------------------------
+// CONSOLE
+// TEST[continued]
+
+Note how we are requesting the concrete rollup index name (`sensor_rollup`) as the first part of the URL.
+This  will yield the following response:
+
+[source,js]
+----
+{
+  "sensor_rollup" : {
+    "rollup_jobs" : [
+      {
+        "job_id" : "sensor",
+        "rollup_index" : "sensor_rollup",
+        "index_pattern" : "sensor-*",
+        "fields" : {
+          "node" : [
+            {
+              "agg" : "terms"
+            }
+          ],
+          "temperature" : [
+            {
+              "agg" : "min"
+            },
+            {
+              "agg" : "max"
+            },
+            {
+              "agg" : "sum"
+            }
+          ],
+          "timestamp" : [
+            {
+              "agg" : "date_histogram",
+              "time_zone" : "UTC",
+              "interval" : "1h",
+              "delay": "7d"
+            }
+          ],
+          "voltage" : [
+            {
+              "agg" : "avg"
+            }
+          ]
+        }
+      }
+    ]
+  }
+}
+----
+// TESTRESPONSE
+
+
+The response that is returned contains information that is similar to the original Rollup configuration, but formatted
+differently.  First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data,
+the index pattern that the job was targeting.
+
+Next it shows a list of fields that contain data eligible for rollup searches.  Here we see four fields: `node`, `temperature`,
+`timestamp` and `voltage`.  Each of these fields list the aggregations that are possible.  For example, you can use a min, max
+or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
+
+Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index
+or index pattern.  Each of these jobs may have different configurations, so the API returns a list of all the various
+configurations available.
+
+
+Like other APIs that interact with indices, you can specify index patterns instead of explicit indices:
+
+[source,js]
+--------------------------------------------------
+GET /*_rollup/_xpack/rollup/data
+--------------------------------------------------
+// CONSOLE
+// TEST[continued]
+

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/action/GetRollupCapsAction.java

@@ -145,7 +145,7 @@ public void writeTo(StreamOutput out) throws IOException {
         public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
             builder.startObject();
             for (Map.Entry<String, RollableIndexCaps> entry : jobs.entrySet()) {
-               entry.getValue().toXContent(builder, params);
+                entry.getValue().toXContent(builder, params);
             }
             builder.endObject();
             return builder;

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/action/GetRollupIndexCapsAction.java

@@ -0,0 +1,201 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.core.rollup.action;
+
+
+import org.elasticsearch.action.Action;
+import org.elasticsearch.action.ActionRequest;
+import org.elasticsearch.action.ActionRequestBuilder;
+import org.elasticsearch.action.ActionRequestValidationException;
+import org.elasticsearch.action.ActionResponse;
+import org.elasticsearch.action.IndicesRequest;
+import org.elasticsearch.action.support.IndicesOptions;
+import org.elasticsearch.client.ElasticsearchClient;
+import org.elasticsearch.common.ParseField;
+import org.elasticsearch.common.Strings;
+import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.common.io.stream.StreamOutput;
+import org.elasticsearch.common.io.stream.Writeable;
+import org.elasticsearch.common.xcontent.ToXContent;
+import org.elasticsearch.common.xcontent.ToXContentObject;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.xpack.core.rollup.RollupField;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Map;
+import java.util.Objects;
+
+public class GetRollupIndexCapsAction extends Action<GetRollupIndexCapsAction.Request,
+    GetRollupIndexCapsAction.Response, GetRollupIndexCapsAction.RequestBuilder> {
+
+    public static final GetRollupIndexCapsAction INSTANCE = new GetRollupIndexCapsAction();
+    public static final String NAME = "indices:data/read/xpack/rollup/get/index/caps";
+    public static final ParseField CONFIG = new ParseField("config");
+    public static final ParseField STATUS = new ParseField("status");
+    private static final ParseField INDICES_OPTIONS = new ParseField("indices_options");
+
+    private GetRollupIndexCapsAction() {
+        super(NAME);
+    }
+
+    @Override
+    public Response newResponse() {
+        return new Response();
+    }
+
+    @Override
+    public RequestBuilder newRequestBuilder(ElasticsearchClient client) {
+        return new RequestBuilder(client, INSTANCE);
+    }
+
+    public static class Request extends ActionRequest implements IndicesRequest.Replaceable, ToXContent {
+        private String[] indices;
+        private IndicesOptions options;
+
+        public Request(String[] indices) {
+            this(indices, IndicesOptions.STRICT_EXPAND_OPEN_FORBID_CLOSED);
+        }
+
+        public Request(String[] indices, IndicesOptions options) {
+            this.indices = indices;
+            this.options = options;
+        }
+
+        public Request() {}
+
+        @Override
+        public IndicesOptions indicesOptions() {
+            return options;
+        }
+
+        @Override
+        public String[] indices() {
+            return indices;
+        }
+
+        @Override
+        public IndicesRequest indices(String... indices) {
+            Objects.requireNonNull(indices, "indices must not be null");
+            for (String index : indices) {
+                Objects.requireNonNull(index, "index must not be null");
+            }
+            this.indices = indices;
+            return this;
+        }
+
+        @Override
+        public void readFrom(StreamInput in) throws IOException {
+            super.readFrom(in);
+            this.indices = in.readStringArray();
+            this.options = IndicesOptions.readIndicesOptions(in);
+        }
+
+        @Override
+        public void writeTo(StreamOutput out) throws IOException {
+            super.writeTo(out);
+            out.writeStringArray(indices);
+            options.writeIndicesOptions(out);
+        }
+
+        @Override
+        public ActionRequestValidationException validate() {
+            return null;
+        }
+
+        @Override
+        public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+            builder.array(RollupField.ID.getPreferredName(), indices);
+            builder.field(INDICES_OPTIONS.getPreferredName(), options);
+            return builder;
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(Arrays.hashCode(indices), options);
+        }
+
+        @Override
+        public boolean equals(Object obj) {
+            if (obj == null) {
+                return false;
+            }
+            if (getClass() != obj.getClass()) {
+                return false;
+            }
+            Request other = (Request) obj;
+            return Arrays.equals(indices, other.indices)
+                && Objects.equals(options, other.options);
+        }
+    }
+
+    public static class RequestBuilder extends ActionRequestBuilder<Request, Response, RequestBuilder> {
+
+        protected RequestBuilder(ElasticsearchClient client, GetRollupIndexCapsAction action) {
+            super(client, action, new Request());
+        }
+    }
+
+    public static class Response extends ActionResponse implements Writeable, ToXContentObject {
+
+        private Map<String, RollableIndexCaps> jobs = Collections.emptyMap();
+
+        public Response() {
+
+        }
+
+        public Response(Map<String, RollableIndexCaps> jobs) {
+            this.jobs = Objects.requireNonNull(jobs);
+        }
+
+        Response(StreamInput in) throws IOException {
+            jobs = in.readMap(StreamInput::readString, RollableIndexCaps::new);
+        }
+
+        public Map<String, RollableIndexCaps> getJobs() {
+            return jobs;
+        }
+
+        @Override
+        public void writeTo(StreamOutput out) throws IOException {
+            super.writeTo(out);
+            out.writeMap(jobs, StreamOutput::writeString, (out1, value) -> value.writeTo(out1));
+        }
+
+        @Override
+        public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+            builder.startObject();
+            for (Map.Entry<String, RollableIndexCaps> entry : jobs.entrySet()) {
+                entry.getValue().toXContent(builder, params);
+            }
+            builder.endObject();
+            return builder;
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(jobs);
+        }
+
+        @Override
+        public boolean equals(Object obj) {
+            if (obj == null) {
+                return false;
+            }
+            if (getClass() != obj.getClass()) {
+                return false;
+            }
+            Response other = (Response) obj;
+            return Objects.equals(jobs, other.jobs);
+        }
+
+        @Override
+        public final String toString() {
+            return Strings.toString(this);
+        }
+    }
+}