VLAN ID is not freed from cache becasuse as EC2 API is evenutally consistent

[abhipth]

Describe the Bug:
When a pod using Branch ENI is created and the controller is immediately restarted since EC2 is eventually consistent the EC2 API call can return ENI list without the newly created ENI leading to the controller not marking the VLAN ID allocated to the ENI as assigned in it's internal cache.

This can lead to new Branch ENI created on the node to fail since the controller will try to reuse the existing VLAN ID which it's not aware of.

Proposed Fix:
Check the error message, if the error message says that the VLAN ID is still in use, add the VLAN ID to cache.

_, err = t.ec2ApiHelper.AssociateBranchToTrunk(&t.trunkENIId, nwInterface.NetworkInterfaceId, vlanID)
	if err != nil {
                // Check error here if VLAN is already used mark it in cache.
		trunkENIOperationsErrCount.WithLabelValues("associate_branch").Inc()
		break
	}

How to reproduce it (as minimally and precisely as possible):
Can be reproduced on trying multiple times to create Branch ENIs for new pods and kill the controller just after new Branch ENIs are created.

Environment:

• Kubernetes version (use kubectl version): v1.0.5
• CNI Version: v1.7.5