Skip to content

8.0.0 released 2019-11-03
Compare
Choose a tag to compare
@jantman jantman released this 03 Nov 23:54
· 190 commits to master since this release
8.0.0
This tag was signed with the committer’s verified signature. The key has expired.
jantman Jason Antman
GPG key ID: 1815A4BC5C0251C9
Expired
Learn about vigilant mode.
b23dd86
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Important: This release includes major changes to the EC2 On-Demand Instances service limits! For most users, this means the 175 Instance-type-specific limits will be removed and replaced with five (5) limits. Please see the changelog.8_0_0_vcpu_limits section below for further details, as this will especially impact anyone using limit or threshold overrides, or post-processing awslimitchecker's output. This is also a time to remind all users that this project adheres to a strict development.versioning_policy and if occasional breakage due to limit or IAM policy changes is unacceptable, you should pin to a major version.

Important: Python versions prior to 3.5, including 2.7, are now pending deprecation. As of January 1, 2020, they will no longer be tested or supported, and awslimitchecker will require Python 3.5 or newer. Please see below for details. Also take note that running via the official Docker image is a way to ensure the best version of Python is always used.

Important: This release requires a new IAM permission, servicequotas:ListServiceQuotas.

  • Issue #400 / PR #434 - Support GovCloud region and alternate partitions in STS assumed roles and Trusted Advisor. Thanks to @djkiourtsis.
  • Issue #432 - Update EC2 limit handling for new vCPU-based limits in regions other than cn-* and us-gov-* (which still use old per-instance-type limits). See section below <changelog.8_0_0_vcpu_limits> for further information. For regions other than cn-* and us-gov-*, this will remove all 175 Running On-Demand <type> instances and the Running On-Demand EC2 instances limit, and replace them with:
    • Running On-Demand All F instances
    • Running On-Demand All G instances
    • Running On-Demand All P instances
    • Running On-Demand All X instances
    • Running On-Demand All Standard (A, C, D, H, I, M, R, T, Z) instances
  • Issue #429 - add 87 missing EC2 instance types. This will now only impact cn-* and us-gov-* regions.
  • Issue #433 - Fix broken links in the docs; waffle.io and landscape.io are both gone, sadly.
  • Issue #441 - Fix critical bug where awslimitchecker would die with an unhandled botocore.exceptions.ParamValidationError exception in accounts that have Trusted Advisor but do not have a "Service Limits" check in the "performance" category.
  • Issue #439 - Fix unhandled exception in CloudTrail service when attempting to call GetEventSelectors on an Organization trail. When calling DescribeTrails, we will now pass includeShadowTrails as False, to not include replications of trails in different regions or organization trails in member accounts (relevant API documentation).
  • Issue #438 - Per PEP 373, Python 2.7 will officially end support on January 1, 2020. As such, and in keeping with reasoning explained at python3statement.org, awslimitchecker will stop supporting and testing against Python 2.7 on January 1, 2020. At that point, all new versions will be free to use Python features introduced in 3.5. As of this version, a PendingDeprecationWarning will be emitted when running awslimitchecker under Python 2.7.
  • Issue #437 - Per PEP 429, Python 3.4 reached end-of-life on March 18, 2019 and is now officially retired. Add a PendingDeprecationWarning for users running under this version, announcing that support for Python 3.4 will be removed on January 1, 2020.
  • In following with the above two issues, raise a DeprecationWarning when running on any Python2 version prior to 2.7 or any Python3 version prior to 3.4, in accorance with the published end-of-life dates of those versions.
  • Issue #436 - Begin testing under Python 3.8 and base our Docker image on python:3.8-alpine.
  • Issue #435 - Allow configuring the botocore maximum retries for Throttling / RateExceeded errors on a per-AWS-API basis via environment variables. See the relevant sections of the CLI Usage or Python Usage documentation for further details.
  • Issue #431 - Fix a major under-calculation of usage for the EC2 Rules per VPC security group limit. We were previously calculating the number of "Rules" (from port / to port / protocol combinations) in a Security Group, but the limit is actually based on the number of permissions granted. See this comment on the issue for further details.
  • Issue #413 - Add support for retrieving limits from the new Service Quotas service where available. See the changelog.8_0_0_service_quotas section below for more information.
  • Bump boto3 minimum version requirement from 1.4.6 to 1.9.175 and botocore minimum version requirement from 1.6.0 to 1.12.175, in order to support Service Quotas.

New EC2 vCPU Limits

AWS has announced new, completely different handling of EC2 On-Demand Instances service limits. Instead of having a limit per instance type (currently 261 limits), there will now be only five limits, based on the number of vCPUs for instance families: one each for "F", "G", "P", and "X" family instances (defaulting to a total of 128 vCPUs each) and one limit for all other "Standard" instance families (currently A, C, D, H, I, M, R, T, and Z) defaulting to a combined total of 1152 vCPUs. Please see the link, and the EC2 On-Demand Instance Limits section of the AWS FAQ for further information.

This greatly simplifies handling of the EC2 On-Demand limits, but does mean that any existing code that references EC2 Running On-Demand limit names, including any limit and/or threshold overrides, will need to be updated for this change.

This change is only going into effect in the "standard" AWS regions/partitions, i.e. not in the China partition (cn- regions) or GovCloud (us-gov- regions). It is a phased rollout from October 24 to November 7, 2019 based on the first character of your account ID (see the "How will the transition to vCPU limits happen?" entry in the FAQ linked above for exact dates). Unfortunately, there is no clear way to determine via API if a given account is using the new vCPU limits or the old per-instance-type limits. As a result, and given that this release is being made already part-way through the rollout window, the current behavior of awslimitchecker is as follows:

  • When running against region names beginning with cn- or us-gov-, use the old per-instance-type limits, unless the USE_VCPU_LIMITS environment variable is set to true.
  • Otherwise use the new vCPU-based limits, unless the USE_VCPU_LIMITS environment variable is set to something other than true.

As such, if you install this release before November 7, 2019 and need to force your non-China, non-GovCloud accout to use the older per-instance-type limits, setting the USE_VCPU_LIMITS environment variable to false will accomplish this until your account switches over to the new vCPU limits. Alternatively, you can leave awslimitchecker as-is and accept possibly-slightly-inaccurate limit calculations for a few days.

Please also note that with the change to vCPU limits, there is no longer an overall Running On-Demand EC2 instances limit for accounts that use the new vCPU limits.

I have not yet implemented Trusted Advisor (TA) support for these new limits, as they're presented in a different category of Trusted Advisor checks from the previous EC2 limits. I'm not going to be implementing TA for these limits, in favor of spending the time instead on implementing Service Quotas support via Issue #413.

Calculation of current usage for the vCPU limits is based on the EC2 Optimizing CPU Options documentation which specifies, "The number of vCPUs for the instance is the number of CPU cores multiplied by the threads per core." The CpuOptions field of the EC2 DescribeInstances API specifies the core and thread count for each running instance.

Service Quotas

AWS' new Service Quotas service provides a unified interface to retrieve current limits from many AWS services. These limit values are second only to the services' own APIs (for the services that provide limit information via API), and are much more current and complete than the information provided by Trusted Advisor. The introduction of Service Quotas should greatly reduce the number of limits that need to be retrieved from Trusted Advisor or specified manually.

If you currently have any Limit Overrides set (via either the CLI or Python API), please verify on the limits page whether Service Quotas data is now available for those limits. You should be able to remove manual overrides for the limits that now retrieve data from Service Quotas.

Assets 2
Loading