New install process (awslabs#677)

* New install process

**Why?**

With the upcoming release of ADF, CDK v2 support is added.
The dependencies that go with it, unfortunately increased the deployment size
beyond the limit that is supported by the Serverless Application Repository.

Hence a new installation mechanism had to be developed.

**What?**

This change proposes to use make as the deployment mechanism.
I chose make, as this eases support queries on deployments of old version and
future versions.

For example, by default, the `make` command will install the latest
`aws-sam-cli` and `yq` dependencies. However, if these appear to be
incompatible, we can instruct to use a specific version of the make install
script:

```bash
make update_makefile

make UPDATE_VERSION=make/2.0 update_makefile
```

Similarly, we can instruct using the AWS SAM CLI used at the time the ADF
version was developed by running:

```bash
make clean deps docker version_number git_ignore sam_build post_build
```

In case someone runs into an issue while building or deploying ADF, they can
generate a report of their environment with:

```bash
make version_report
```

This generates a report of all the versions of the build environment.

_Long story short:_
The proposed Makefile enables various paths to resolve and investigate issues.

**Changes**

* Updated the Makefile to act as the new deployment mechanism.
* Moved tox related tasks to test and lint the code to Makefile.tox.
  As these need to run in the virtual environment managed by Tox, not the
  one managed by the new Makefile.
* Updated the docs to support ADF administrators updating and installing
  using the new Makefile.
* Tested with v3.2.0 deployments and the upcoming release of ADF.
* Added support to generate version numbers on customized ADF installations.
* Updated the contribution docs to clarify how to deploy and test.
* Added support to update the Makefile in the future, such that a deployment
  issue in a specific ADF version can be fixed by updating the Makefile.
* Updated GitHub workflows accordingly, plus added new tests to ensure
  the dependencies installed by ADF are compatible with each other.
* Added a new step in the GitHub workflow to test a full build of ADF.
* Updated the Bug Issue template to include the version report and to allow
  the reporter to indicate that they are working on a fix already.

* Add macOS support to generate a version report

* Update contribution guide to include upstream tags fetch step

* Fix contributing guide typo

.editorconfig

@@ -51,7 +51,7 @@ indent_size = 4
 [tox.ini]
 max_line_length = 120
 
-[Makefile]
+[Makefile*]
 indent_style = tab
 indent_size = 4
 tab_width = 4

.github/ISSUE_TEMPLATE/bug-template.yaml

@@ -59,7 +59,7 @@ body:
     attributes:
       label: Possible Solution
       description: |
-        Suggest a fix/reason for the bug
+        Suggest a fix/reason for the bug.
     validations:
       required: false
   - type: textarea
@@ -70,6 +70,12 @@ body:
         Anything else that might be relevant for troubleshooting this bug.
         Providing context helps us come up with a solution that is most useful
         in the real world.
+
+        If you are facing a build or deployment issue, please include details
+        about the environment you are working in. You can run the following
+        command to capture all relevant details in one report:
+
+        make version_report
     validations:
       required: false
   - type: input
@@ -79,3 +85,11 @@ body:
       description: The ADF version you are running.
     validations:
       required: true
+  - type: checkboxes
+    attributes:
+      label: Contributing a fix?
+      description: >
+        Are you are able to contribute and resolve this issue?
+      options:
+        - label: Yes, I am working on a fix to resolve this issue
+          required: false

.github/workflows/adf.yml

@@ -19,10 +19,13 @@ jobs:
       - name: Set the correct Node version using nvm
         shell: bash -l {0}
         run: nvm install
-      - name: Install dependencies
-        run: make init
+      - name: Test compatibility of source dependencies
+        run: |
+          make clean src_deps
+          echo "Source dependencies are compatible!"
       - name: Run tox
-        # Run tox using the version of Python in `PATH`
         run: |
-          tox --version
-          tox
+          make clean tox
+      - name: Build fully
+        run: |
+          make clean build

.gitignore

@@ -18,9 +18,12 @@ master-deploy.yml
 shared_layer.zip
 .aws-sam
 samconfig.toml
+samconfig.yml
+samconfig.yaml
 pipeline.json
 template-sam.yml
 deploy.sh
+Makefile.new
 
 # Byte-compiled / optimized / DLL files
 __pycache__

CONTRIBUTING.md

@@ -16,33 +16,50 @@ features.
 When filing an issue, please check
 [existing open](https://github.com/awslabs/aws-deployment-framework/issues),
 and [recently closed](https://github.com/awslabs/aws-deployment-framework/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aclosed%20)
-issues to make sure somebody else hasn't already reported the issue.
+issues to make sure somebody else has not already reported the issue.
 
 Please try to include as much information as you can.
 Details like these are incredibly useful:
 
 - A reproducible test case or series of steps
 - The version of our code being used
+- For deployment related bugs, please include the details returned by the
+  following command as executed on the development machine:
+  `make report_versions`
 - Any modifications you have made relevant to the bug
 - Anything unusual about your environment or deployment
 
 ## Running tests locally
 
-In order to run the tests locally you need a virtual environment that is used
-by [tox](https://pypi.org/project/tox/).
+In order to run the tests locally you need install the development requirements
+in the python virtual environment that is used.
 
-1. Create a virtual environment: `virtualenv .tox/env`
-2. Activate the virtual environment: `source .tox/env/bin/activate`
-3. Install dependencies: `make init`
-4. To run the tests, execute: `tox`
+To ensure test execute consistently, ADF relies on
+[tox](https://pypi.org/project/tox/).
+
+The easiest way to install and run these tests is to run:
+
+```bash
+make tox
+```
+
+Alternatively, you can also install the required dependencies in your
+python virtual environment by running:
+
+```bash
+pip install -r requirements-dev.txt
+```
+
+To run the tests, simply execute next: `tox`.
+This will create a virtual environment managed by tox to run the tests in.
 
 ## Running linters locally
 
 You need to have NodeJS and Docker installed on your computer to run MegaLinter
 locally with MegaLinter Runner.
 
-You can run mega-linter-runner without installation by using `npx` (Run from the
-root of the repository!).
+You can run mega-linter-runner without installation by using `npx`.
+Make sure to execute this from the root of the repository.
 
 ```sh
 npx mega-linter-runner
@@ -54,6 +71,23 @@ Some linters can automatically fix findings by running the command below.
 npx mega-linter-runner --fix
 ```
 
+## Deploy your changes
+
+1. To deploy your changes, make sure to commit them in your local repository
+   first.
+2. If you are working on your own fork, make sure to add the upstream
+   repository as a remote and fetch its tags. As these tags are used to
+   generate the version number. You do this by:
+
+   ```bash
+   git remote add upstream https://github.com/awslabs/aws-deployment-framework
+   git fetch upstream --tags
+   ```
+
+3. Once you committed and optionally fetched the upstream repo tags, follow the
+   [installation guide](./docs/installation-guide.md) to deploy them into your
+   management account.
+
 ## Contributing via Pull Requests
 
 Contributions via pull requests are much appreciated.
@@ -121,6 +155,39 @@ following account ids may be used:
 - `012345671234`
 - `123456789012`
 
+## Resolving Build and deployment issues
+
+Please capture the environment the build/deployment issue occurs in:
+
+```bash
+make version_report
+```
+
+Before you report an issue, please try again after cleaning the environment:
+
+```bash
+make clean
+```
+
+Possibly, this issue is fixed already, please update your Makefile and try
+again after:
+
+```bash
+make update_makefile
+```
+
+If that did not resolve the issue, please try running:
+
+```bash
+make clean deps_build
+```
+
+Or download an older version of the Makefile by running:
+
+```bash
+make UPDATE_VERSION=make/2.0 update_makefile
+```
+
 ## Code of Conduct
 
 This project has adopted the