dklb
supports exposing TCP services only.
In particular, exposing UDP or SCTP services is NOT supported.
To expose a TCP application running on MKE to either inside or outside the DC/OS cluster, a Kubernetes Service
resource of type LoadBalancer
must be created.
dklb
will react to this event by provisioning an EdgeLB pool (henceforth referred to as the target EdgeLB pool) for the Service
resource.
This EdgeLB pool is provisioned using sane default values for its name, its CPU, memory and size requirements, and other options.
After provisioning said EdgeLB pool, dklb
will periodically query EdgeLB in order to obtain the list of hostnames and IPs at which the service can be reached.
These will eventually be reported on the .status
field of the Service
resource.
It should be noted that, due to the way EdgeLB pool scheduling and metadata reporting works, it may take from a few seconds to several minutes for these hostnames and IPs to be reported.
As mentioned before, dklb
uses sane defaults when provisioning EdgeLB pools for Service
resources of type LoadBalancer
.
It is, however, possible to customize the target EdgeLB pool for a given Service
resource by using the kubernetes.dcos.io/dklb-config
annotation.
This annotation accepts a YAML object (henceforth called the configuration object) with the following structure:
kubernetes.dcos.io/dklb-config: | # NOTE: The "|" character is mandatory.
name: "dklb-pool-0"
role: "*"
network: "dcos"
size: 3
cpus: 0.2
memory: 512
frontends:
- port: 18080
servicePort: 8080
- port: 19090
servicePort: 9090
The subsections below provide more insight on each of the fields on the configuration object.
ℹ️
|
The |
|
The kubernetes.dcos.io/dklb-config cannot be removed after the Service resource is created.
|
By default, dklb
uses the MKE cluster’s name and a randomly-generated five-character suffix to compute the name of the target EdgeLB pool.
To specify a custom name for said EdgeLB pool, one may use the .name
field of the configuration object:
kubernetes.dcos.io/dklb-config: |
name: "<edgelb-pool-name>"
Depending on whether the <edgelb-pool-name>
EdgeLB pool exists or not, dklb
will create or update it in order to expose all ports defined in the Service
resource.
❗
|
This field cannot be changed or removed after the Service resource is created.
|
By default, dklb
exposes services to outside the DC/OS cluster by requesting for the target EdgeLB pool to be scheduled onto a public DC/OS agent.
However, and in order to accommodate all possible needs, dklb
supports explicitly specifying a Mesos role for the target EdgeLB pool using the .role
field of the configuration object:
kubernetes.dcos.io/dklb-config: |
role: "<edgelb-pool-role>"
In particular, to expose a service to inside DC/OS only, *
should be used as the value of <edgelb-pool-role>
.
Providing said value will cause dklb
to request for the target EdgeLB pool to be scheduled onto a private DC/OS agent.
❗
|
This field cannot be changed or removed after the Service resource is created.
|
dklb
provisions the target EdgeLB pool by looking at the service’s ports and creating an EdgeLB backend and an EdgeLB frontend per port.
By default, dklb
uses the port number that is defined on the Service
resource as the frontend’s bind port.
In some situations, however, using a different port number as the frontend’s bind port may be required.[1]
In order to accommodate more advanced use cases, dklb
supports defining custom port mappings via the .frontends
field of the configuration object:
kubernetes.dcos.io/dklb-config: |
frontends:
- port: <frontend-bind-port>
servicePort: <service-port>
When such an item is provided on the .frontends
field, dklb
will use <frontend-bind-port>
instead of <service-port>
as the actual frontend bind port.
|
Changing the value of a One should plan port mappings ahead whenever possible in order to prevent changes from being required in the first place. |
dklb
supports customizing CPU, memory and size requests for the target EdgeLB pool.
Custom values for these requests can be specified using the .cpus
, .memory
and .size
fields, respectively:
kubernetes.dcos.io/dklb-config: |
cpus: <edgelb-pool-cpus>
memory: <edgelb-pool-memory>
size: <edgelb-pool-size>
In the above representation, <edgelb-pool-cpus>
is a floating-point number (e.g. 0.2
), and <edgelb-pool-memory>
and <edgelb-pool-size>
are integers (e.g. 512
and 3
, respectively).
dklb
supports customizing load balancer instance placement for the target EdgeLB Pool.
By default, no constraint is specified. A custom value can be specified using the constraints
field.
|
Take special care to escape strings in the placement constraint. |
kubernetes.dcos.io/dklb-config: |
contraints: "<Marathon style constraints for load balancer instance placement>"
By design, pools exposing Kubernetes services to outside the DC/OS cluster (i.e. pools using the slave_public
role) will run atop the DC/OS agents' host network.
Also by design, pools exposing Kubernetes services to inside the DC/OS cluster will run atop the dcos
virtual network.
It is, however, possible to override this and pick any https://docs.mesosphere.com/1.12/networking/SDN/usage/ [custom DC/OS virtual network] for these pools by using the .network
field of the configuration object:
kubernetes.dcos.io/dklb-config: |
network: "<edgelb-pool-network>"
❗
|
This field cannot be changed or removed after the Service resource is created.
|
In certain scenarios, it may be desirable to use a pre-existing EdgeLB pool to expose a Kubernetes service (instead of having dklb
creating one).
This can easily be achieved by providing the name of the pre-existing EdgeLB pool as the value of the .name
field of the configuration object.
To share an EdgeLB pool between two or more Kubernetes services, it is enough to provide the name of said pool as the value of the .name
field of the configuration object in all of the corresponding Service
resources.
When an EdgeLB pool is shared between two or more Kubernetes services, the following aspects should be taken into consideration:
-
The
.role
,.network
,.cpus
,.memory
and.size
fields must have the exact same value across allService
resources sharing an EdgeLB pool. -
Sharing an EdgeLB pool between services in different MKE clusters is allowed, but should be avoided whenever possible.
-
Changing or deleting one of the
Service
resources exposed on a shared EdgeLB pool may cause disruption in all applications exposed on said EdgeLB pool.
This example illustrates how to expose a Redis instance to outside the DC/OS cluster.
To start with, a simple redis
pod will be created:
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
labels:
app: redis
name: redis
spec:
containers:
- name: redis
image: redis:5.0.3
ports:
- name: redis
containerPort: 6379
protocol: TCP
EOF
pod/redis created
$ kubectl get pod --selector "app=redis"
NAME READY STATUS RESTARTS AGE
redis 1/1 Running 0 100s
Then, a Service
resource of type LoadBalancer
targeting the specified pod will also be created:
$ cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Service
metadata:
annotations:
kubernetes.dcos.io/dklb-config: |
name: dklb-redis
frontends:
- port: 16379
servicePort: 6379
labels:
app: redis
name: redis
spec:
type: LoadBalancer
selector:
app: redis
ports:
- protocol: TCP
port: 6379
targetPort: 6379
EOF
service/redis created
$ kubectl get svc --selector "app=redis"
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis LoadBalancer 10.100.80.96 <pending> 6379:32213/TCP 2m42s
The kubernetes.dcos.io/dklb-config
defined on this Service
resource will cause dklb
to expose the service using an EdgeLB pool called dklb-redis
, mapping the service’s 6379
port to the EdgeLB pool’s 16379
port.
At this point, querying the EdgeLB API should confirm the existence of a pool called dklb-redis
exposing said port:
$ dcos edgelb list
NAME APIVERSION COUNT ROLE PORTS
dklb-redis V2 1 slave_public 9090, 16379
This means that dklb
has successfully created and provisioned the target EdgeLB pool based on the spec of the redis
service.
To test connectivity, it is necessary to determine the public IP at which the target EdgeLB pool can be reached.
This IP will eventually be reported in the .status
field of the Service
resource:
$ kubectl get svc --selector "app=redis"
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
redis LoadBalancer 10.100.80.96 <public-dcos-agent-ip> 6379:32213/TCP 2m42s
telnet
may then be used to confirm that the Redis instance is correctly exposed to outside the DC/OS cluster:
$ telnet <public-dcos-agent-ip> 16379
Trying <public-dcos-agent-ip>...
Connected to <public-dcos-agent-ip>.
Escape character is '^]'.
|
Depending on the firewall rules in place for the DC/OS cluster, it may be necessary to manually allow traffic to port 16379 in order to allow connectivity.
|
This means that the Redis instance is indeed reachable at <public-dcos-agent-ip>:16379
(i.e., from outside the DC/OS cluster).
Additional commands may be run inside telnet
in order to verify that everything is working as expected:
$ telnet <public-dcos-agent-ip> 16379
(...)
SET foo bar
+OK
GET foo
$3
bar
QUIT
+OK
Connection closed by foreign host.