Support for multiple weighted clusters in routing (#361)

docs/configuration/http_conn_man/route_config/route.rst

@@ -12,6 +12,7 @@ next (e.g., redirect, forward, rewrite, etc.).
     "prefix": "...",
     "path": "...",
     "cluster": "...",
+    "weighted_clusters" : "...",
     "host_redirect": "...",
     "path_redirect": "...",
     "prefix_rewrite": "...",
@@ -39,8 +40,17 @@ path
 
 cluster
   *(sometimes required, string)* If the route is not a redirect (*host_redirect* and/or
-  *path_redirect* is specified), *cluster* must be specified and indicates which upstream cluster
-  the request should be forwarded to.
+  *path_redirect* is not specified), one of *cluster* or *weighted_clusters* must be specified.
+  When a *cluster* is specified, its value indicates the upstream cluster to which the request
+  should be forwarded to.
+
+:ref:`weighted_clusters <config_http_conn_man_route_table_route_weighted_clusters>`
+  *(sometimes required, string)* If the route is not a redirect (*host_redirect* and/or
+  *path_redirect* is not specified), one of *cluster* or *weighted_clusters* must be specified. 
+  With the *weighted_clusters* option, multiple upstream clusters can be specified for a given route.
+  The request is forwarded to one of the upstream clusters based on weights assigned
+  to each cluster. See :ref:`traffic splitting <config_http_conn_man_route_table_weighted_routing>`
+  for additional documentation.
 
 .. _config_http_conn_man_route_table_route_host_redirect:
 
@@ -216,3 +226,52 @@ regex
 The router will check the request's headers against all the specified
 headers in the route config. A match will happen if all the headers in the route are present in
 the request with the same values (or based on presence if the ``value`` field is not in the config).
+
+.. _config_http_conn_man_route_table_route_weighted_clusters:
+
+Weighted Clusters
+-----------------
+
+Compared to the ``cluster`` field that specifies a single upstream cluster as the target
+of a request, the ``weighted_clusters`` option allows for specification of multiple upstream clusters
+along with weights that indicate the **percentage** of traffic to be forwarded to each cluster.
+The router selects an upstream cluster based on the weights.
+
+.. code-block:: json
+
+   {
+     "clusters": [],
+     "runtime_key_prefix" : "..."
+   }
+
+clusters
+  *(required, array)* Specifies one or more upstream clusters associated with the route. 
+
+  .. code-block:: json
+
+     {
+       "name" : "...",
+       "weight": "..."
+     }
+
+  name
+    *(required, string)* Name of the upstream cluster. The cluster must exist in the
+    :ref:`cluster manager configuration <config_cluster_manager>`.
+
+  weight
+    *(required, integer)* An integer between 0-100. When a request matches the route,
+    the choice of an upstream cluster is determined by its weight. The sum of
+    weights across all entries in the ``clusters`` array must add up to 100.
+
+runtime_key_prefix
+  *(optional, string)* Specifies the runtime key prefix that should be used to construct the runtime
+  keys associated with each cluster. When the ``runtime_key_prefix`` is specified, the router will
+  look for weights associated with each upstream cluster under the key 
+  ``runtime_key_prefix + "." + cluster[i].name`` where ``cluster[i]``  denotes an entry in the
+  ``clusters`` array field. If the runtime key for the cluster does not exist, the value specified
+  in the configuration file will be used as the default weight.
+  See the :ref:`runtime documentation <operations_runtime>` for how key names map to the
+  underlying implementation.
+
+  **Note:** If the sum of runtime weights exceed 100, the traffic splitting behavior
+  is undefined (although the request will be routed to one of the clusters).

docs/configuration/http_conn_man/route_config/route_config.rst

@@ -56,3 +56,4 @@ response_headers_to_remove
   rate_limits
   route_matching
   traffic_shifting
+  weighted_routing

docs/configuration/http_conn_man/route_config/weighted_routing.rst

@@ -0,0 +1,120 @@
+.. _config_http_conn_man_route_table_weighted_routing:
+
+Traffic splitting across multiple upstreams
+===========================================
+
+Envoy's router can split traffic to a route in a virtual host across
+multiple upstream clusters. There are two common use cases. The first use
+case is version upgrades, where traffic to a route is shifted gradually
+from one cluster to another. The
+:ref:`traffic shifting <config_http_conn_man_route_table_traffic_shifting>`
+describes this scenario in more detail.
+
+The second use case, discussed in this section, is A/B testing or multivariate
+testing, where ``two or more versions`` of the same service are tested simultaneously.
+The traffic to the route has to be *split* between clusters running different versions
+of the same service.
+
+Consider a simple example ``service`` with three versions (v1, v2 and
+v3). Envoy provides two ways to split traffic evenly across the three
+versions (i.e., ``33%, 33%, 34%``):
+
+.. _config_http_conn_man_route_table_weighted_routing_percentages:
+
+(a) Weight-based cluster selection
+----------------------------------
+
+The first option is to use a **single** :ref:`route <config_http_conn_man_route_table_route>` with 
+:ref:`weighted_clusters <config_http_conn_man_route_table_route_weighted_clusters>`,
+where multiple upstream cluster targets are specified for a single route,
+along with weights that indicate the **percentage** of traffic to be sent
+to each upstream cluster.
+
+.. code-block:: json
+
+    {
+      "route_config": {
+        "virtual_hosts": [
+          {
+            "name": "service",
+            "domains": ["*"],
+            "routes": [
+              {
+                "prefix": "/",
+                "weighted_clusters": {
+                  "runtime_key_prefix" : "routing.traffic_split.service",
+                  "clusters" : [
+                    { "name" : "service_v1", "weight" : 33 },
+                    { "name" : "service_v2", "weight" : 33 },
+                    { "name" : "service_v3", "weight" : 34 }
+                  ]
+                }
+              }
+            ]
+          }
+        ]
+      }
+    }
+
+The weights assigned to each cluster can be dynamically adjusted using the
+following runtime variables: ``routing.traffic_split.service.service_v1``,
+``routing.traffic_split.service.service_v2`` and
+``routing.traffic_split.service.service_v3``.
+
+.. _config_http_conn_man_route_table_weighted_routing_probabilities:
+
+(b) Probabilistic route selection
+---------------------------------
+
+The second option is to use **multiple** :ref:`routes <config_http_conn_man_route_table_route>`
+with :ref:`runtimes <config_http_conn_man_route_table_route_runtime>` that specify the
+**probability** of selecting a route.
+Since Envoy matches routes with a :ref:`first match <config_http_conn_man_route_table_route_matching>`
+policy, the related routes (one for each upstream cluster) must be placed back-to-back,
+along with a runtime in all but the last route.
+
+.. code-block:: json
+
+    {
+      "route_config": {
+        "virtual_hosts": [
+          {
+            "name": "service",
+            "domains": ["*"],
+            "routes": [
+              {
+                "prefix": "/",
+                "cluster": "service_v1",
+                "runtime": {
+                  "key": "routing.traffic_split.service.service_v1",
+                  "default": 33
+                }
+              },
+              {
+                "prefix": "/",
+                "cluster": "service_v2",
+                "runtime": {
+                  "key": "routing.traffic_split.service.service_v2",
+                  "default": 50
+                }
+              },
+              {
+                "prefix": "/",
+                "cluster": "service_v3",
+              }
+            ]
+          }
+        ]
+      }
+    }
+
+In the configuration above,
+
+1. ``routing.traffic_split.service.service_v1`` is set to ``33``, so that there is a
+   *33\% probability* that the v1 route will be selected by Envoy.
+2. ``routing.traffic_split.service.service_v2`` is set to ``50``, so that if the v1 route
+   is not selected, between v2 and v3, there is a *50\% probability* that the v2 route will
+   be selected by Envoy. If v2 is not selected the traffic falls through to the v3 route.
+
+This distribution of probabilities ensures that the traffic will be split evenly across
+all three routes (i.e. ``33%, 33%, 34%``).