diff --git a/README.md b/README.md index 6eb876d..ea67825 100644 --- a/README.md +++ b/README.md @@ -15,11 +15,15 @@ the user running the resource counting script: - `roles/owner` - `roles/cloudasset.owner` - `roles/cloudasset.viewer` - - [Security Command Center][2] (faster, but not free). The user running the - script must have one of these roles at the organization-level: - - `roles/resourcemanager.organizationAdmin` - - `roles/securitycenter.admin` - - `roles/securitycenter.adminViewer` + - This method has been deprecated. As outlined in + [Listing assets using the Security Command Center API][4], + this functionality has been deprecated on `June 20, 2023` and will reach its + EOL on `June 0, 2024`. + ~~[Security Command Center][2] (faster, but not free). The user running the + script must have one of these roles at the organization-level:~~ + - ~~`roles/resourcemanager.organizationAdmin`~~ + - ~~`roles/securitycenter.admin`~~ + - ~~`roles/securitycenter.adminViewer`~~ 3. Install [jq][3] 4. Retrieve the GCP organization ID by running the following command and looking for the ID of the organization to count: @@ -46,26 +50,22 @@ from the previous step: { "appengine.googleapis.com/Application": 20, "cloudfunctions.googleapis.com/CloudFunction": 63, - "compute.googleapis.com/Image": 43, "compute.googleapis.com/Instance": 466, + "compute.googleapis.com/K8RelatedInstance": 8, "sqladmin.googleapis.com/Instance": 65, - "storage.googleapis.com/Bucket": 367 + "storage.googleapis.com/Bucket": 367, + "k8s.io/Node": 8, } ``` -## How to run using security command center (gcloud_scc_inventory.sh) +### Note about output -```bash -λ organizationID=123456789012 ./gcloud_scc_inventory.sh -{ - "appengine.googleapis.com/Application": 20, - "cloudfunctions.googleapis.com/CloudFunction": 63, - "compute.googleapis.com/Image": 43, - "compute.googleapis.com/Instance": 466, - "sqladmin.googleapis.com/Instance": 65, - "storage.googleapis.com/Bucket": 367 -} -``` +`compute.googleapis.com/K8RelatedInstance` is not an actual asset that +listed in [Supported asset types][5]. The number generated for this +custom asset is from filterings all compute instances that have a +`goog-gke-node` label. This label is used because it's a protected +and automatically applied label to compute instances that were created +by a GKE cluster. ## Troubleshooting @@ -79,24 +79,9 @@ null Verify IAM for user running asset inventory script has one of the [roles required](#requirements) and has access to the organization. -### Running security command center returns error - - -```bash -ERROR: (gcloud.scc.assets.group) User [your-user-id@your-organization] does not -have permission to access organizations instance [123456789012] (or it may not -exist): Permission 'securitycenter.assets.group' denied on resource -'//cloudresourcemanager.googleapis.com/organizations/123456789012' (or it may -not exist) -``` - - -[Verify security command center is enabled](4) for the organization. Also verify -IAM for user running security command center script has one of the -[roles required](#requirements) and has access to the organization. - [0]: https://cloud.google.com/sdk/docs/install-sdk [1]: https://cloud.google.com/asset-inventory/docs/listing-assets [2]: https://cloud.google.com/security-command-center/docs/set-up [3]: https://stedolan.github.io/jq/download/ -[4]: https://console.cloud.google.com/security/command-center/overview +[4]: https://cloud.google.com/security-command-center/docs/how-to-api-list-assets +[5]: https://cloud.google.com/asset-inventory/docs/supported-asset-types#searchable_asset_types diff --git a/gcloud_asset_inventory.sh b/gcloud_asset_inventory.sh index 1cb2ead..93ca470 100755 --- a/gcloud_asset_inventory.sh +++ b/gcloud_asset_inventory.sh @@ -25,8 +25,10 @@ declare -ar types=( appengine.googleapis.com/Application cloudfunctions.googleapis.com/CloudFunction compute.googleapis.com/Instance + compute.googleapis.com/K8RelatedInstance sqladmin.googleapis.com/Instance storage.googleapis.com/Bucket + k8s.io/Node ) check_binary "gcloud" @@ -41,16 +43,30 @@ outfile=$(mktemp -q) # shellcheck disable=SC2048 for type in ${types[*]}; do - (echo "${organizations[*]}" \ - | xargs -n1 -I{} gcloud asset list \ - --asset-types "$type" \ - --content-type resource \ - --format 'value(assetType)' \ - --organization {} \ - --snapshot-time "$time" \ - | sort \ - | uniq -c \ - | awk '{ printf "{\"%s\":%s}\n", $2, $1 }' >> "$outfile") & + if [ "$type" == "compute.googleapis.com/K8RelatedInstance" ]; then + (echo "${organizations[*]}" \ + | xargs -n1 -I{} gcloud asset list \ + --asset-types "compute.googleapis.com/Instance" \ + --content-type resource \ + --format 'value(assetType)' \ + --organization {} \ + --snapshot-time "$time" \ + --filter 'resource.data.labels:goog-gke-node' \ + | sort \ + | uniq -c \ + | awk '{ printf "{\"compute.googleapis.com/K8RelatedInstance\":%s}\n", $1 }' >> "$outfile") & + else + (echo "${organizations[*]}" \ + | xargs -n1 -I{} gcloud asset list \ + --asset-types "$type" \ + --content-type resource \ + --format 'value(assetType)' \ + --organization {} \ + --snapshot-time "$time" \ + | sort \ + | uniq -c \ + | awk '{ printf "{\"%s\":%s}\n", $2, $1 }' >> "$outfile") & + fi done diff --git a/gcloud_scc_inventory.sh b/gcloud_scc_inventory.sh deleted file mode 100755 index 9778ed2..0000000 --- a/gcloud_scc_inventory.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -check_binary(){ - if ! command -v "${1}" &> /dev/null - then - echo "${1} is a required binary, please install" - exit 1 - fi -} - -declare -r organizationID=${organizationID:-} - -declare -r time=${1:-$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")} - -check_binary "gcloud" -check_binary "jq" - -if [ -z "$organizationID" ]; then - echo "usage: organizationID=someID gcloud_scc_inventory.sh" - exit 1 -fi - -gcloud scc assets group "${organizationID}" \ - --format json \ - --group-by "security_center_properties.resource_type" \ - --page-size 1000 \ - --read-time "$time" \ - | jq -rS '[.[0].groupByResults[] | { "\(.properties[])": .count }] | add'