Documenting etcd infrastructure and establish basic controls

[jmhbnz]

This pull request gets some initial documentation in place collating information that used to spread across a variety of github issues.

Additionally this pull request proposes some very basic improvements to how we control access to the infrastructure. Below is a summary of the proposal:

Current state	Proposed state
No source of truth for who should actually have access to the servers.	Create a new markdown table as a source of truth we can maintain.
Users ssh directly to the machines as root user which is a security anti pattern.	Create individual users for each person needing access.
Everyone has root access just by connecting to a server, with no second factor of authentication required.	Gate root usage behind sudo with a password so users need to have a second factor (key + sudo password) for root. We need to do this because we can't enforce that users set a password for their ssh key meaning if a threat actor gets hold of a users key they could have free reign on etcd infrastructure.
No formal access reviews being performed.	Establish a clear responsibility that access reviews be performed and define a process for revoking access.

In future we are likely to improve this further by shifting towards more as code management of etcd infrastructure, however I believe it is important we at least get some basic processes in place while we work on long term improvements.

Fixes: #15952

[jmhbnz]

Hey @dims - Keen for your thoughts on this proposal given your work earlier this year getting initial servers created.

[dims]

@jmhbnz +100 from me on all points. A simple infrastructure.md with list of folks with access (with their github id, slack id, email) is something we can start with. we need this to be well-curated (drop inactives for examples). the same page can list the machines / ip addresses and other details that this group have rights to administer. The same group can "talk" to CNCF or Equinix or others as needed. So basically start simple and tweak as you grow.

[jmhbnz]

@jmhbnz +100 from me on all points. A simple infrastructure.md with list of folks with access (with their github id, slack id, email) is something we can start with.

Thanks for the feedback @dims, I have updated to a simple markdown table list to get us started, and proposed a starting list of two most active maintainers along with @chaochn47 and I who both have been helping with the infra recently, and then yourself as a CNCF representative.

@ahrtr, @serathius - Can you please let me know if we should be adding anyone else to the list for now? I don't mind maintaining it in future as we iterate.

[serathius]

Can we make it a separate file in some format that could be used for automation? There is no benefit of displaying this here to readers of this doc.

[jmhbnz]

My initial suggestion was to use a new github team called infra-admins. That would unblock the work that @hh and I wanted to do together for an actions workflow that will run a snippet of terraform for automatically creating infra.

We can also then use github api's for other automation. What do you think?

[ahrtr]

I have no strong opinion here. Ideally, moving the list to a separate file and automating the process (e.g. grant access or revoke access) is good, but we will most likely not update the list very often. So putting the list here is simple and good enough for now, and we can update it in future when needed.

[jmhbnz]

I think initially let's get these basic controls merged so I can start implementation, I'm keen to continue contributing in this area and iterating on this in future.

[ahrtr]

Overall looks good to me. Thanks @jmhbnz .

[ahrtr]

Nice work, thanks again @jmhbnz .

Leave it to @serathius to approve and merge.