Enabling reusing probe connection in activator

[julz]

When a pod shows up in endpoints, the activator probes it with the K-Network-Probe header to establish readiness before it proxies over requests. Since the probe (before this change) did not share a transport (and therefore a keep-alive connection pool) with the proxy, the probe connections couldn't be reused for the actual requests, meaning we're ending up pointlessly doing the (relatively expensive) connection dance twice on pod startup.

This change shares the transport (and therefore the keep-alive pool) between the throttler and the activator handler, allowing probe connections to be reused for actual requests, and avoiding the extra reconnect.

Note: before and after this change all transports are created via pkg/network.NewAutoTransport, so the actual transport options should be the same (other than the keep alive pool size and the fact it's now being shared) after this PR as before.

Enables connection reuse between activator probe and proxy to improve first request performance after cold starts

/assign @markusthoemmes @vagababov

[knative-prow-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: julz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cmd/activator/OWNERS [julz]
• pkg/activator/OWNERS [julz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

Merging #10228 (56631ac) into master (45afb29) will increase coverage by 0.02%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master   #10228      +/-   ##
==========================================
+ Coverage   88.13%   88.15%   +0.02%     
==========================================
  Files         184      184              
  Lines        8606     8606              
==========================================
+ Hits         7585     7587       +2     
+ Misses        786      784       -2     
  Partials      235      235              

Impacted Files	Coverage Δ
cmd/activator/main.go	0.00% <0.00%> (ø)
pkg/activator/net/throttler.go	88.29% <0.00%> (ø)
pkg/reconciler/revision/reconcile_resources.go	80.72% <0.00%> (-2.41%)	⬇️
pkg/activator/net/revision_backends.go	91.40% <0.00%> (+0.90%)	⬆️
pkg/reconciler/configuration/configuration.go	88.28% <0.00%> (+1.56%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 45afb29...c884d5a. Read the comment docs.

[vagababov]

We need both transports. Since for the case when there's mesh and we're talking to the k8s service we historically needed one that would be recreated on each requests.

[julz]

the mesh case already uses a connection with keep-alive for probing here - that isn't changed by this PR. Also, Im pretty sure we only need to avoid keep-alive for the stats scraping case where we want to retry and get different backends each time: in activator even for the mesh case we just wait for it to be healthy at all in checkDests.

[vagababov]

/lgtm

[julz]

/retest

[markusthoemmes]

Good one!