Flapping HostIP when node has multiple IP addresses

[Reamer]

What happened:
Using Openshift with vsphere cloud provider, works as expected till you add multiple egressIPs. After adding multiple EGRESS IPS the cloud provider changes the HOST IP randomly to a IP which is available on node. With this behavior we get routing problems inside the Openshift-cluster.

Normal and expected state:

11:20 $ oc get hostsubnet
NAME                                 HOST                                 HOST IP       SUBNET           EGRESS CIDRS       EGRESS IPS
s-cp-infra-01.cloud.mycompany.com    s-cp-infra-01.cloud.mycompany.com    10.20.15.31   10.128.0.0/23    [10.20.15.96/28]   []
s-cp-infra-02.cloud.mycompany.com    s-cp-infra-02.cloud.mycompany.com    10.20.15.32   10.129.0.0/23    [10.20.15.96/28]   []
s-cp-infra-03.cloud.mycompany.com    s-cp-infra-03.cloud.mycompany.com    10.20.15.33   10.130.14.0/23   [10.20.15.96/28]   [10.20.15.102, 10.20.15.98, 10.20.15.99, 10.20.15.100, 10.20.15.97, 10.20.15.101, 10.20.15.103]
s-cp-master-01.cloud.mycompany.com   s-cp-master-01.cloud.mycompany.com   10.20.15.11   10.131.0.0/23    []                 []
...

s-cp-infra-03.cloud.mycompany.com changes the IP address nearly every second which one IPs available trough EGRESS IPS and the HOST IP

NAME                                 HOST                                 HOST IP       SUBNET           EGRESS CIDRS       EGRESS IPS
s-cp-infra-01.cloud.mycompany.com    s-cp-infra-01.cloud.mycompany.com    10.20.15.31   10.128.0.0/23    [10.20.15.96/28]   []
s-cp-infra-02.cloud.mycompany.com    s-cp-infra-02.cloud.mycompany.com    10.20.15.32   10.129.0.0/23    [10.20.15.96/28]   []
s-cp-infra-03.cloud.mycompany.com    s-cp-infra-03.cloud.mycompany.com    10.20.15.99   10.130.14.0/23   [10.20.15.96/28]   [10.20.15.102, 10.20.15.98, 10.20.15.99, 10.20.15.100, 10.20.15.97, 10.20.15.101, 10.20.15.103]
s-cp-master-01.cloud.mycompany.com   s-cp-master-01.cloud.mycompany.com   10.20.15.11   10.131.0.0/23    []                 []
.. 

What you expected to happen:
host IP should be stable

How to reproduce it (as minimally and precisely as possible):

• Install Openshift
• Configure VSphere Cloudprovider
• Add add least seven EGRESS IPS

Anything else we need to know?:

• I didn't noticed any problem with disc attach and detach

Environment:

• Kubernetes version (use kubectl version):

oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://s-openshift.mycompany.com:443
openshift v3.11.0+6c2b013-59
kubernetes v1.11.0+d4cacc0

• Cloud provider or hardware configuration:

[Global]
    user = "XXXXX@vsphere.local"
    password = "XXXXXXX"
    port = 443
    insecure-flag = 1
    datacenter = "OCP-Datacenter"

[VirtualCenter "10.4.4.102"]

[Workspace]
    server = "10.4.4.102"
    datacenter = "OCP-Datacenter"
    folder = "/OCP-Datacenter/vm/"
    default-datastore = "iscsi-hdd"

[Disk]
    scsicontrollertype = pvscsi

[Network]
    public-network = "n-vlan-openshift-staging-pub"

• OS (e.g. from /etc/os-release):

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

• Kernel (e.g. uname -a):

Linux s-cp-master-01.cloud.mycompany.com 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux