Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update contributing document #1627

Merged
merged 15 commits into from
Nov 1, 2022
Merged

Update contributing document #1627

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
5ea3e1b
update-contributing-document
qiaozha Oct 20, 2022
b1ae2bd
add contributing md
qiaozha Oct 21, 2022
3c87132
update to resolve some comments
qiaozha Oct 21, 2022
f064066
rename file
qiaozha Oct 21, 2022
003734d
update resolve comments
qiaozha Oct 21, 2022
509635b
update to resolve comments
qiaozha Oct 21, 2022
8a97ab9
Update packages/rlc-common/CONTRIBUTING.md
qiaozha Oct 24, 2022
3a54ec3
merge main
qiaozha Oct 25, 2022
6cdf10d
merge main
qiaozha Oct 27, 2022
3c2cadc
update docs organization
qiaozha Oct 27, 2022
bcc4dff
update contributing guide
qiaozha Oct 28, 2022
3895033
update to resolve comments
qiaozha Nov 1, 2022
698e9e0
update resolve comments
qiaozha Nov 1, 2022
a17302c
update contributing guide
qiaozha Nov 1, 2022
e67c67d
fix format
qiaozha Nov 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 75 additions & 1 deletion .vscode/launch.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,60 @@
"${workspaceFolder}/packages/*/dist-dev/**/*.js"
]
},
{
"type": "node",
"request": "launch",
"name": "[autorest.typescript]-UnitTests",
"cwd": "${workspaceFolder}/packages/autorest.typescript",
"program": "./node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--timeout",
"999999",
"--colors",
"./test/unit/**/*.spec.ts"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"protocol": "inspector"
},
{
"type": "node",
"request": "launch",
"name": "[autorest.typescript]-IntegrationTests",
"cwd": "${workspaceFolder}/packages/autorest.typescript",
"program": "./node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--timeout",
"999999",
"--colors",
"./test/integration/**/*.spec.ts"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"protocol": "inspector"
},
{
"type": "node",
"request": "launch",
"name": "[autorest.typescript]-IntegrationTests - Current File",
"cwd": "${workspaceFolder}/packages/autorest.typescript",
"program": "./node_modules/mocha/bin/_mocha",
"args": [
"-r",
"ts-node/register",
"--timeout",
"999999",
"--colors",
"${workspaceFolder}/${relativeFile}"
],
"console": "integratedTerminal",
"internalConsoleOptions": "neverOpen",
"protocol": "inspector"
},
{
"name": "Generate code for CadlRlcTest",
"request": "launch",
Expand All @@ -48,7 +102,27 @@
"outFiles": ["${workspaceFolder}/packages/*/dist/**/*.js"]
},
{
"name": "Cadl Unit or Integration Test",
"name": "[CADL] - Unit Test",
"request": "launch",
"cwd": "${workspaceFolder}/packages/cadl-typescript",
"runtimeArgs": [
"mocha",
"-r",
"ts-node/register",
"--experimental-specifier-resolution=node",
"--experimental-modules=true",
"--timeout",
"999999",
"--colors",
"./test/unit/**/*.spec.ts"
],
"runtimeExecutable": "npx",
"skipFiles": ["<node_internals>/**"],
"type": "pwa-node",
"outFiles": ["${workspaceFolder}/packages/*/dist/**/*.js"]
},
{
"name": "[CADL] - Integration Test",
"request": "launch",
"cwd": "${workspaceFolder}/packages/cadl-typescript",
"runtimeArgs": [
Expand Down
195 changes: 12 additions & 183 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,189 +33,18 @@ If your contribution is significantly big it is better to first check with the p

Want to get started hacking on the code? Great! Keep reading. This contributing guide helps you to start working with and contributing to the Azure Typescript/Javascript SDK Generator.

### General Autorest generation design
## Prerequisites

_TBA_
In order contribute to this project, You will need to install some prerequisite dependencies to get start.

### Typescript/Javascript SDK generator Design

The generated Typescript/Javascript SDK has the following structure:

1. Client (Handled by [clientFileGenerator.ts](./src/generators/clientFileGenerator.ts))
2. Client Context (Handled by [clientContextFileGenerator.ts](./src/generators/clientContextFileGenerator.ts))
3. Operation Interfaces (Handled by [operationInterfaceGenerator.ts](./src/generators/operationInterfaceGenerator.ts))
4. Operations (Handled by [operationGenerator.ts](./src/generators/operationGenerator.ts))
5. Models (Handled by [modelsGenerator.ts](./src/generators/modelsGenerator.ts))
6. Parameters (Handled by [parametersGenerator.ts](./src/generators/parametersGenerator.ts))
7. Index File (Handled by [indexGenerator.ts](./src/generators/indexGenerator.ts))
8. Metadata (Handled [here](./src/generators/static))

The entry point for this design is at [main.ts](./src/main.ts) which invokes the [typescriptGenerator.ts](./src/typescriptGenerator.ts) and starts the entire generation process. You could find all the options that could be sent to the generator at [autorestOptions.ts](./src/utils/autorestOptions.ts)

On a high level, the entire SDK generation process would be:

Swagger Input -> Autorest Core & Modeler -> CodeModel -> TS/JS SDK Generator -> Generated SDK

Within the TS/JS SDK Generator, it has the following stages:

CodeModel + User Options -> Transform CodeModel -> Generate Client -> Generate Client Context -> Generate Operations/Interfaces -> Generate Models -> Generate Parameters -> Generate Index -> Format & Output Generated SDK

### Steps to clone, build & test

1. Use the following command to clone the Typescript/Javascript SDK generator repository:

```
git clone https://github.com/Azure/autorest.typescript.git
```

2. Use the following commands to build the SDK generator:

```
npm install -g @autorest/autorest
cd autorest.typescript
npm install
npm run build
```

3. There are 3 test-suites in the generator:
1. Unit tests (which could be found at `test/unit/*`)
2. Integration tests (which could be found at `test/integration/*`)
3. Smoke tests
4. Version Tolerence Tests
4. You can run the Unit tests & Integration tests using the following command:

```
npm run test
```

Running the command above will do the following things:

- Start TestServer
- Build Autorest.Typescript
- Generate all scenarios in parallel (i.e. BodyString, BodyComplex, Url, CustomUrl, Header, XmlService
- Run all the tests under test/integration
- Stop TestServer

**_Note_**: If your development environment is Windows, then run the command `npm run start-test-server:v2`(in a seperate window) before running `npm run test` and run the command `npm run stop-test-server` after. (In non windows machines, we could run the test-server in the background automatically. But, in Windows machines, it has to be done manually.)

5. You can run the Smoke tests using the following command:

```
npm run clone:specs
npm run smoke-test
```

**_Note_**: If the command `npm run clone:specs` errors out with the following error:

```
fatal: destination path './.tmp/specs' already exists and is not an empty directory.
```

delete the `.tmp` folder and then try again.

### Version Tolerence Tests

Version tolerance tests provide coverage against unexpected **breaking changes** when generating a newer version of a swagger. RLCs are not expected to produce breaking changes unless the swagger itself has an API breaking change.

Version Tolerance tests would generate 2 clients one from an initial Swagger and a second from the `updated` swagger that contains changes that are not expected to generate breaking changes. The same set of tests is run against both generated clients to make sure no breaking changes resulted.

Version Tolerance tests have the following npm scripts:

- `npm run validate-version-tolerance`: generates SDKs and tests for breaking changes between swagger versions
- `generate-version-tolerance`: Generates version tolerance before and after sdks
- `generate-version-tolerance:tests`: generates `updated` spec.ts based on hand written tests against the initial SDK
- `test-version-tolerance`: executes all .spec.ts tests under `test/version-tolerance/`

In CI we'll run `npm run validate-version-tolerance` which calls all the other npm scripts. The other individual npm scripts are useful for development.

For more details on the implementation see [#1268](https://github.com/Azure/autorest.typescript/pull/1268)

### How to add an integration test case

Whenever you work on adding a feature/fixing a bug, this would probably be your first step. You create a test case and then run it through the generator, see the result, modify the generator, run it again and so on, until you get the desired output.

1. Create your test input. Below are some examples
1. A simple [Swagger/OpenAPI file](./test/integration/swaggers/license-header.json))
2. A standalone [configuration markup file](./test/integration/swaggers/bodyComplex.md) (pointing to Swagger file somewhere else)
3. A combination of configuration markup file and Swagger/OpenAPI file.

Let us say your test input will be called `testUserCase.json`.

2. Now add the test input to the integration test suite to the file [`test-swagger-gen.ts`](./test/commands/test-swagger-gen.ts). In the file, add the following to the array `testSwaggers`:

```
testUserCase: {
swaggerOrConfig: "test/integration/swaggers/testUserCase.json",
clientName: "TestUserCaseClient",
packageName: "test-user-case",
licenseHeader: true,
useCoreV2: true,
allowInsecureConnection: true,
addCredentials: false
},
```

3. Now, You can generate the SDK only for your test case with the following command: (Initially, during your development, you do not want to run all the cases during every step of your development. But, once your code changes are complete for your case, then you need to run the entire suite to ensure that your changes did not cause any unwanted changes.)

```
npm run generate-swaggers -- -b -i testUserCase
```

4. Once you are satisfied with the generated code, you can add a spec file such as `testUserCase.spec.ts` file [here](./test/integration). You can find several examples in the same place.

### How to debug an integration test case

#### `generate-swaggers` step

If you would like to debug the `generate-swaggers` step for our test input, use the following command:

```
npm run generate-swaggers -- -b -i testUserCase --debug
```

Now, the code will wait for the debugger to be attached. Open the repository in VS Code -> Select `Run and Debug` section -> Click `Attach`.

#### Spec file

If you would like to debug the `testUserCase.spec.ts` file (after the SDK is generated), Open the repository in VS Code -> Open the `testUserCase.spec.ts` file -> Select `Run and Debug` section -> Click `IntegrationTests - Current File`.

#### How to debug an unit test case

- In VS Code, We have created a Debugging profile for UnitTests to start debugging:

1. Go to the debugger tab
2. Select the "Unit Test" Profile
3. Click the "Play" button

- Your breakpoints will start hitting, you can set breakpoints in either Test or Generator code

#### Integration Tests

- In order to debug integration tests you need to start the test server, by running:

npm run start-test-server:v1

- Once the Test Server is running

1. In VSCode go to the debugger tab
2. Select the "IntegrationTests" profile from the drop down
3. Click the "Play" button

- **\*\***IMPORTANT**\*\***: Running Integration Tests for debugging, does not re-generate the test clients so make sure that after each change you do:

- Re-generate all the test swaggers

npm run generate-swaggers -- --build

- Re-generate a specific swagger

npm run generate-swaggers -- -i bodyComplex -b

### Before investing more time investigating

- Make sure to reset autorest, this will ensure that the versions used by Auotrest Core are the correct ones. This will solve many problems

autorest --reset
- Git
- Any of the [LTS versions of Node.js](https://github.com/nodejs/release#release-schedule)
- Rush 5.x
- Install / update Rush globally via `npm install -g @microsoft/rush`.
- Rush will automatically manage the specific version needed by this repo as long as you have any v5 version installed.
- If you're unable to install a global tool, you can instead call the wrapper script `node <repo root>/common/scripts/install-run-rush.js` any time the guide instructs you to run `rush`. The wrapper script will install a managed copy of Rush in a temporary directory for you.
- [Autorest](https://www.npmjs.com/package/autorest), if you're planning contribute to the generator code from swagger either for high level client or for rest level client.
- [Cadl Compiler](https://www.npmjs.com/package/@cadl-lang/compiler), if you're planning contribute to the generator code from cadl.

### Things to keep in mind when contributing

Expand All @@ -231,7 +60,7 @@ If your test case is working fine as expected, now you are ready to create the P

Our generated SDKs take dependency on various packages which you can see in the generated package.json files. These will need to be upgraded from time to time to stay on the latest major version so that we get bug fixes automatically due to semver.

- Update the version of the dependency you are looking for in the methods `restLevelPackage` and/or `regularAutorestPackage` in the [`packageFileGenerator.ts`](https://github.com/Azure/autorest.typescript/blob/main/src/generators/static/packageFileGenerator.ts) file.
- Run `npm run build && npm run generate-swaggers && npm run smoke-tests` to update the generated files in the repo and run smoke tests before creating the PR.
- Update the version of the dependency you are looking for in the methods `restLevelPackage` and/or `regularAutorestPackage` in the [`packageFileGenerator.ts`](https://github.com/Azure/autorest.typescript/blob/main/packages/autorest.typescript/src/generators/static/packageFileGenerator.ts) file.
- Run `rush rebuild && npm run generate-swaggers && npm run smoke-tests` to update the generated files in the repo and run smoke tests before creating the PR.

For example, see the [PR 1271](https://github.com/Azure/autorest.typescript/pull/1271) that updates the version of `prettier` that the generated SDKs depend on.
30 changes: 30 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Azure TypeScript SDK and RLC Code Generator Libraries

This repository is the code generator libraries for Azure TypeScript SDK and Azure TypeScript Rest Level Client libraries.

## Packages

| Name | Changelog | Latest | Next |
| --------------------------------------------------------------------------- | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------- |
| [@autorest/typescript][autorest_typescript_src] | [Changelog][autorest_typescript_chg] | [![](https://img.shields.io/npm/v/@autorest/typescript)](https://www.npmjs.com/package/@autorest/typescript) | ![](https://img.shields.io/npm/v/@autorest/typescript/next) |
| [@azure-tools/cadl-typescript][cadl-typescript_src] | [Changelog][cadl-typescript_chg] | [![](https://img.shields.io/npm/v/@azure-tools/cadl-typescript)](https://www.npmjs.com/package/@azure-tools/cadl-typescript) | ![](https://img.shields.io/npm/@azure-tools/cadl-azure-core/next) |
| [@azure-tools/rlc-common][rlc-common_src] | [Changelog][rlc-common_chg] | [![](https://img.shields.io/npm/v/@azure-tools/rlc-common)](https://www.npmjs.com/package/@azure-tools/rlc-common) | ![](https://img.shields.io/npm/@azure-tools/rlc-common/next) |


[autorest_typescript_src]: packages/autorest.typescript/
[autorest_typescript_chg]: packages/autorest.typescript/CHANGELOG.md
[cadl-typescript_src]: packages/cadl-typescript
[cadl-typescript_chg]: packages/cadl-typescript/CHANGELOG.md
[rlc-common_src]: packages/rlc-common
[rlc-common_chg]: packages/rlc-common/CHANGELOG.md


`@next` version of the package are the latest versions available on the `main` branch.

### General Introduction

Inside this project, we support both high level client generation and rest level client generation. In the rest level client generation, we support generate code from both rest api specs and cadl. In the high level client generation, we only support generation from rest api specs.

- [**@autorest/typescript:**](https://github.com/Azure/autorest.typescript/tree/main/packages/autorest.typescript) contains the original high level client generation and the rest level client generation, mostly including the transformation from autorest code model into RLC model part.
- [**@azure-tools/cadl-typescript:**](https://github.com/Azure/autorest.typescript/tree/main/packages/cadl-typescript) is the typescript cadl emitter, which contains the transformation from cadl model into RLC model.
- [**@azure-tools/rlc-common:**](https://github.com/Azure/autorest.typescript/tree/main/packages/rlc-common) contains the common generation logic from RLC model into rest client libraries code part. Which is depended by both `@autorest/typescript` rlc generation part and `@azure-tools/cadl-typescript`.
Loading