Merge from master to firebase (#159)

* Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect (#38)

* Simple TCP server to show how to retrieve original dest IP:port after an iptables redirect

* Fixed style.

* Rebase Envoy (#41)

* Update prototype to use iptables (#42)

* Rebase to fixed Envoy (#43)

* Handle HEAD request. (#34)

* Handle HEAD request.

* Try with GET if HEAD fails.

* Address comments.

* Format file.

* Expose bazel target (#48)

* Try again (#49)

* Integrate with mixer client. (#55)

* Integrate with mixer client.

* Restore  repositories.bzl back.

* Add originIp and originHost attributes. (#56)

* Add uuid-dev dependency in README.md (#45)

* Extract originIp and OriginHost. (#57)

* Extract originIp and OriginHost.

* Make header x-forwarded-host const.

* Update buckets for UI. (#58)

* Update buckets for UI.

* Only update time_distribution.

* Add targetService attribute. (#59)

* Use envoy new access_log handler for sending Report. (#60)

* use access_log handler.

* Not to use Loggable base class.

* Update to the latest envoy with #396. (#61)

* Fix tclap dependency fetching error (#62)

* Integrate mixer client directly with envoy. (#66)

* Integrate mixer client directly with envoy.

* Send response header in Report.

* rename filter name from esp to mixer.

* add README.

* Add release binary script. (#68)

* Push tar.gz to GCS (#69)

* Push tar.gz to GCS

* Rename envoy_esp

* Remove mixer_client from api_manager. (#72)

* Update mixer client SHA. (#74)

* Update readme. (#73)

* Adds Jenkinsfile and updates release-binary to create a SHA. (#71)

* Adds Jenkinsfile and update release-binary
* Update Jenkinsfile and gitignore
* Fixes typo and use normal build Node
* Uses default bazel config
* Using batch mode
* Update bazel memory settings
* Do not use Jenkins bazel env
* Set .bazelrc for postsubmit

* Update grpc and protobuf (#70)

* protobuf v3.2.0
* grpc v1.1.1
* Align auth lib with grpc 1.1.1

* Add sourceService. (#78)

* Add script to build docker image. (#77)

* Add script to build docker image.

* Add start_envoy for docker image.

* Use official attribute names (#80)

* Use official attribute names

* fix format

* Creates a KEY for mixer client dep. Updates release-binary (#79)

* Updated mixer repo to use a key for commit

* release-binary skip build if file exists.

* Update src/envoy/mixer/README. (#82)

* Fix src/envoy/mixer/README.md (#85)

* Get attributes from envoy config. (#87)

* Send all attributes.

* Remove unused const strings.

* Address comment.

* updated SHA to point to newer envoy with RDS API feature (#94)

* Disable travis on stable branches (#96)

* Publish debug binaries (no release yet) (#98)

* Copies the binary instead of linking for release (#102)

* Not to use api_key if its service is not actived. (#109)

* Update envoy and add c-ares (#107)

* Update envoy and add c-ares depedencies

* Update release script with debug and normal binary

* remove debug ls

* formatting

* Send StatusCode Attributes to Mixer. (#110)

* Add send_attribute filter. (#115)

* Add send_attribute filter.

* Fix format

* rename variable serialized_attributes_

* Address the comments.

* Fail request if api_key is not valid. (#116)

* Fail request if api_key is not valid.

* Format code.

* Update comments.

* Address comment.

* Rename response.http.code (#125)

* Send headers as string map. (#129)

* Send headers as string map.

* Remove origin.ip and origin.host.

* Fix format

* unify bazel's docker build targets with other istio repos (#127)

* update base debug docker image reference (#133)

* Update postsubmit to create docker images (#132)

* Adding config release for bazel build (#135)

* Fix mixer client crash. (#136)

* Get mixerclient with response parsing. (#138)

* Update nghttp2 to sync with envoy (#140)

* Fix src/envoy/mixer/README.md

* Update nghttp2 to sync with envoy

* update

* fix typo

* Populate origin.user attribute from the SAN field of client cert (#142)

* Test

* test

* test

* revert file

* address comments

* test

* fix typo

* fix format

* fix format

* Update to latest mixer_client. (#145)

* Update to latest mixer_client.

* Updated the sha.

* Not call report if decodeHeaders is not called. (#150)

* Update mixerclient with sync-ed grpc write and fail-fast. (#155)

* Update mixerclient with sync-ed write and fail-fast.

* Update to latest test.

* Update again

* Update envoy to PR553 (#156)

* Update envoy to PR553

* Update libevent to 2.1.8

* Update the Commit id for envoy

src/envoy/mixer/http_control.cc

@@ -31,59 +31,24 @@ namespace Mixer {
 namespace {
 
 // Define attribute names
-const std::string kRequestPath = "request.path";
+const std::string kOriginUser = "origin.user";
+
+const std::string kRequestHeaders = "request.headers";
 const std::string kRequestHost = "request.host";
+const std::string kRequestPath = "request.path";
 const std::string kRequestSize = "request.size";
 const std::string kRequestTime = "request.time";
-const std::string kRequestHeaders = "request.headers";
 
 const std::string kResponseHeaders = "response.headers";
+const std::string kResponseHttpCode = "response.http.code";
+const std::string kResponseLatency = "response.latency";
 const std::string kResponseSize = "response.size";
 const std::string kResponseTime = "response.time";
-const std::string kResponseLatency = "response.latency";
-const std::string kResponseHttpCode = "response.http.code";
-
-Attributes::Value StringValue(const std::string& str) {
-  Attributes::Value v;
-  v.type = Attributes::Value::STRING;
-  v.str_v = str;
-  return v;
-}
-
-Attributes::Value StringMapValue(
-    std::map<std::string, std::string>&& string_map) {
-  Attributes::Value v;
-  v.type = Attributes::Value::STRING_MAP;
-  v.string_map_v.swap(string_map);
-  return v;
-}
-
-Attributes::Value Int64Value(int64_t value) {
-  Attributes::Value v;
-  v.type = Attributes::Value::INT64;
-  v.value.int64_v = value;
-  return v;
-}
-
-Attributes::Value TimeValue(
-    std::chrono::time_point<std::chrono::system_clock> value) {
-  Attributes::Value v;
-  v.type = Attributes::Value::TIME;
-  v.time_v = value;
-  return v;
-}
-
-Attributes::Value DurationValue(std::chrono::nanoseconds value) {
-  Attributes::Value v;
-  v.type = Attributes::Value::DURATION;
-  v.duration_nanos_v = value;
-  return v;
-}
 
 void SetStringAttribute(const std::string& name, const std::string& value,
                         Attributes* attr) {
   if (!value.empty()) {
-    attr->attributes[name] = StringValue(value);
+    attr->attributes[name] = Attributes::StringValue(value);
   }
 }
 
@@ -103,39 +68,43 @@ void FillRequestHeaderAttributes(const HeaderMap& header_map,
                                  Attributes* attr) {
   SetStringAttribute(kRequestPath, header_map.Path()->value().c_str(), attr);
   SetStringAttribute(kRequestHost, header_map.Host()->value().c_str(), attr);
-  attr->attributes[kRequestTime] = TimeValue(std::chrono::system_clock::now());
+  attr->attributes[kRequestTime] =
+      Attributes::TimeValue(std::chrono::system_clock::now());
   attr->attributes[kRequestHeaders] =
-      StringMapValue(ExtractHeaders(header_map));
+      Attributes::StringMapValue(ExtractHeaders(header_map));
 }
 
 void FillResponseHeaderAttributes(const HeaderMap* header_map,
                                   Attributes* attr) {
   if (header_map) {
     attr->attributes[kResponseHeaders] =
-        StringMapValue(ExtractHeaders(*header_map));
+        Attributes::StringMapValue(ExtractHeaders(*header_map));
   }
-  attr->attributes[kResponseTime] = TimeValue(std::chrono::system_clock::now());
+  attr->attributes[kResponseTime] =
+      Attributes::TimeValue(std::chrono::system_clock::now());
 }
 
 void FillRequestInfoAttributes(const AccessLog::RequestInfo& info,
                                int check_status_code, Attributes* attr) {
   if (info.bytesReceived() >= 0) {
-    attr->attributes[kRequestSize] = Int64Value(info.bytesReceived());
+    attr->attributes[kRequestSize] =
+        Attributes::Int64Value(info.bytesReceived());
   }
   if (info.bytesSent() >= 0) {
-    attr->attributes[kResponseSize] = Int64Value(info.bytesSent());
+    attr->attributes[kResponseSize] = Attributes::Int64Value(info.bytesSent());
   }
 
   if (info.duration().count() > 0) {
-    attr->attributes[kResponseLatency] = DurationValue(
+    attr->attributes[kResponseLatency] = Attributes::DurationValue(
         std::chrono::duration_cast<std::chrono::nanoseconds>(info.duration()));
   }
 
   if (info.responseCode().valid()) {
     attr->attributes[kResponseHttpCode] =
-        Int64Value(info.responseCode().value());
+        Attributes::Int64Value(info.responseCode().value());
   } else {
-    attr->attributes[kResponseHttpCode] = Int64Value(check_status_code);
+    attr->attributes[kResponseHttpCode] =
+        Attributes::Int64Value(check_status_code);
   }
 }
 
@@ -170,8 +139,9 @@ void HttpControl::FillCheckAttributes(HeaderMap& header_map, Attributes* attr) {
 }
 
 void HttpControl::Check(HttpRequestDataPtr request_data, HeaderMap& headers,
-                        DoneFunc on_done) {
+                        std::string origin_user, DoneFunc on_done) {
   FillCheckAttributes(headers, &request_data->attributes);
+  SetStringAttribute(kOriginUser, origin_user, &request_data->attributes);
   log().debug("Send Check: {}", request_data->attributes.DebugString());
   mixer_client_->Check(request_data->attributes, on_done);
 }

src/envoy/mixer/http_control.h

@@ -42,7 +42,7 @@ class HttpControl final : public Logger::Loggable<Logger::Id::http> {
 
   // Make mixer check call.
   void Check(HttpRequestDataPtr request_data, HeaderMap& headers,
-             ::istio::mixer_client::DoneFunc on_done);
+             std::string origin_user, ::istio::mixer_client::DoneFunc on_done);
 
   // Make mixer report call.
   void Report(HttpRequestDataPtr request_data,

src/envoy/mixer/http_filter.cc

@@ -19,6 +19,7 @@
 #include "common/http/headers.h"
 #include "common/http/utility.h"
 #include "envoy/server/instance.h"
+#include "envoy/ssl/connection.h"
 #include "server/config/network/http_connection_manager.h"
 #include "src/envoy/mixer/http_control.h"
 #include "src/envoy/mixer/utils.h"
@@ -151,8 +152,16 @@ class Instance : public Http::StreamFilter, public Http::AccessLog::Instance {
     state_ = Calling;
     initiating_call_ = true;
     request_data_ = std::make_shared<HttpRequestData>();
+
+    std::string origin_user;
+    Ssl::Connection* ssl =
+        const_cast<Ssl::Connection*>(decoder_callbacks_->ssl());
+    if (ssl != nullptr) {
+      origin_user = ssl->uriSanPeerCertificate();
+    }
+
     http_control_->Check(
-        request_data_, headers,
+        request_data_, headers, origin_user,
         wrapper([this](const Status& status) { completeCheck(status); }));
     initiating_call_ = false;
 
@@ -180,6 +189,7 @@ class Instance : public Http::StreamFilter, public Http::AccessLog::Instance {
     }
     return FilterTrailersStatus::Continue;
   }
+
   void setDecoderFilterCallbacks(
       StreamDecoderFilterCallbacks& callbacks) override {
     Log().debug("Called Mixer::Instance : {}", __func__);
@@ -227,6 +237,8 @@ class Instance : public Http::StreamFilter, public Http::AccessLog::Instance {
                    const HeaderMap* response_headers,
                    const AccessLog::RequestInfo& request_info) override {
     Log().debug("Called Mixer::Instance : {}", __func__);
+    // If decodeHaeders() is not called, not to call Mixer report.
+    if (!request_data_) return;
     // Make sure not to use any class members at the callback.
     // The class may be gone when it is called.
     // Log() is a static function so it is OK.

src/envoy/mixer/repositories.bzl

@@ -15,7 +15,7 @@
 ################################################################################
 #
 
-MIXER_CLIENT = "5b5745f29ac5a8babe79ada573defaa83f3bb9e7"
+MIXER_CLIENT = "92a305961bcea90ed349ffdbb0ea299c6f6bacad"
 
 def mixer_client_repositories(bind=True):
     native.git_repository(

src/envoy/repositories.bzl

@@ -62,6 +62,7 @@ event_srcs = [
     "evthread.c",
     "evutil.c",
     "evutil_rand.c",
+    "evutil_time.c",
     "http.c",
     "listener.c",
     "log.c",
@@ -86,9 +87,7 @@ cc_library(
         "defer-internal.h",
         "evbuffer-internal.h",
         "event-internal.h",
-        "event.h",
         "evthread-internal.h",
-        "evutil.h",
         "http-internal.h",
         "iocp-internal.h",
         "ipv6-internal.h",
@@ -134,8 +133,8 @@ cc_library(
 
     native.new_http_archive(
         name = "libevent_git",
-        url = "https://github.com/libevent/libevent/releases/download/release-2.0.22-stable/libevent-2.0.22-stable.tar.gz",
-        strip_prefix = "libevent-2.0.22-stable",
+        url = "https://github.com/libevent/libevent/releases/download/release-2.1.8-stable/libevent-2.1.8-stable.tar.gz",
+        strip_prefix = "libevent-2.1.8-stable",
         build_file_content = BUILD,
     )
 
@@ -752,6 +751,6 @@ cc_test(
     native.new_git_repository(
         name = "envoy_git",
         remote = "https://github.com/lyft/envoy.git",
-        commit = "70e5d651b55d356770529e5bee9c6b2707d9cf21", # 3/1/2017
+        commit = "9dcac8ca111ecc8da059d1f8d42eb766b44bacd6", # https://github.com/lyft/envoy/pull/553
         build_file_content = BUILD,
     )