Extend VPC (x86_64) Infrastructure reconciliation support

[cjschaef]

/kind feature
/area provider/ibmcloud

Describe the solution you'd like
The current implementation for reconciling VPC (x86_64) Infrastructure support is very basic and lacks full feature/resource support for creating VPC Resources to be used for cluster Infrastructure. Expending the support for VPC to reconcile more resources and in more complex configurations can help with a more complete set of Infrastructure for clusters.

NOTE: Implementation for extended Machine reconciliation will be covered separately

Anything else you would like to add:
In order to prevent breaking existing (legacy) functionality, a new code path (a v2 path) will be developed, for more complex field parsing, configuration and resource definitions. We'd expect to eventually migrate all VPC Infrastructure reconciliation to this new path and deprecate and remove the old functionality.

---

Expected Functional changes

Major items identified, but not limited to:

• New API's
  • Multiple Subnets
  • Custom Image
  • Security Group and Security Group Rules
  • Load Balancer resources (pools, etc.)
• New Cluster Controller Reconciliation for
  • VPC
  • Custom Image
  • Subnets
  • Security Groups and Security Group Rules
  • Load Balancers

Expected Prerequisites

• Existing Resource Group (RG for any existing resources and the Cluster's Resource Group)
• Existing COS Instance, Bucket, Object containing qcow image for VPC Custom Image generation (or provided existing VPC Custom Image)
• IAM Authorization for VPC to COS for creating VPC Custom Image using COS Object

Expected Resource Specifications

• Existing resources can be designated by Name or ID (where applicable)
• Resources specified by ID that don't exist or cannot be found results in an error
• Resources specified by Name will trigger a lookup (treated as existing), or get created if not found (some limitations include ResourceGroup, which will not be created regardless of how it was designated)
• Certain resources are dependent on other resources, requiring a hierarchy of reconciliation/creation, but also, limits the way a resource is specified, even if it is expected to exist (runtime resource ID's are not available prior to Infrastructure creation, meaning names may be designated instead, but if those resources don't already exist at that point of resource reconciliation, an error will result rather than try to create). Some examples of this: SecurityGroupRules can depend on Subnet CIDR's or other SecurityGroups which must already exist; Load Balancers can have Subnets and SecurityGroups attached during creation, so those resources must already exist

[mkumatag]

cc @Karthik-K-N @dharaneeshvrd

[mkumatag]

@cjschaef can you please update this task with all of your PRs delivered so far to items you have listed above?

[cjschaef]

PR Tracking

API Changes

• VPC: Add Security Group resources and constants/enums
• Fix kubebuilder validation for SecurityGroup resources
• Update PortRange and ICMP types for Security Groups
• VPC: Extend VPC API's to support v2 Infra
• VPC: Add v2 support for VPC reconcile
• VPC Infrastructure: Add v2 support for remaining VPC Infrastructure resources

Functional Changes

• VPC: Create v2 path for new Infrastructure implementation
• VPC: Extend VPC related API's for v2 VPC reconcile
• VPC: Add Custom Image reconciliation
• VPC: Subnet reconcile
• TBD: Security Group/SG Rule reconcile
• TBD: Load Balancer reconcile