update api scenario documentation (Azure#16086)

* add operation step * fix schema * restore OpTest * add cleanUpSteps & remove type * use draft-07 schema * refine propertyNames * add description * remove merge * fix copy/move * use property name * variable type * restore plain style * define Example and file reference in v1.1 * remove raw Example support * fix * rename to API scenario * restore entry * fix ref
AzureSDKAutomation · Oct 11, 2021 · 3a40099 · 3a40099
1 parent 0e27f38
commit 3a40099
Show file tree

Hide file tree

Showing 19 changed files with 573 additions and 496 deletions.
diff --git a/...tation/test-scenario/how-to/QuickStart.md → ...ntation/api-scenario/how-to/QuickStart.md b/...tation/test-scenario/how-to/QuickStart.md → ...ntation/api-scenario/how-to/QuickStart.md
@@ -14,14 +14,15 @@
 ```sh
 npm install -g oav@latest
 ```
+
 ### OAV Features
+
 - Very easy to use and run.
 - Support postman collection format. Debug easily.
 - Request response validation. `oav` implement a powerful validation algorithm and help developer to detect service issue in the early phase.
-- Validation result report. After each run test scenario, developer will get a validation report which contains detect issue in api test.
+- Validation result report. After each run API scenario, developer will get a validation report which contains detect issue in api test.
 - Integrate everywhere. Easily integrate with azure-pipeline, cloud-test.
 
-
 ## Create AAD app
 
 To run API test, first please prepare an AAD app which is used for provisioning Azure resource. Please grant subscription contributor permission to this AAD app.
@@ -30,24 +31,24 @@ For how to create AAD app, please follow this doc https://docs.microsoft.com/en-
 
 ## Authoring steps
 
-We will write test scenario file for SignalR service as an example.
+We will write API scenario file for SignalR service as an example.
 
-#### 1. Write your first test scenario file
+#### 1. Write your first API scenario file
 
-First, create a folder `scenarios` under the api version folder. All test scenario files under the `scenarios` folder should bind with the api version.
+First, create a folder `scenarios` under the api version folder. All API scenario files under the `scenarios` folder should bind with the api version.
 
 ![folder-structure](./folder-structure.png)
 
-Now write your basic test scenario. For more detail about test scenario file format, please refer to
-[Test Scenario Definition Reference](../references/TestDefinitionReference.md).
+Now write your basic API scenario. For more detail about API scenario file format, please refer to
+[API Scenario Definition Reference](../references/ApiScenarioDefinition.md).
 
 ```yaml
-# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/test-scenario/references/v1.0/schema.json
+# yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-rest-api-specs/main/documentation/api-scenario/references/v1.1/schema.json
 
-contentVersion: 1.0.0
 scope: ResourceGroup
-testScenarios:
-  - description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
+scenarios:
+  - scenario: quickStart
+    description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
     steps:
       - step: SignalR_CreateOrUpdate
         exampleFile: ../examples/SignalR_CreateOrUpdate.json
@@ -57,7 +58,7 @@ testScenarios:
 
 #### 2. create your env file
 
-The `env.json` file contains required test scenario variables such as, subscriptionId, AAD applicationId, AAD applicationSecret.
+The `env.json` file contains required API scenario variables such as, subscriptionId, AAD applicationId, AAD applicationSecret.
 
 ```json
 {
@@ -77,9 +78,9 @@ oav run /home/user/azure-rest-api-specs/specification/signalr/resource-manager/M
 
 #### 4. Debug with postman
 
-Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the test scenario with postman.
+Sometimes the command `oav run` may fail due to non 2xx HTTP status code. Now you need to debug the API scenario with postman.
 
-When run `run`, it automatically generate postman collection and postman env in `generated/<providerNamespace>/<testScenarioFile>/<runId>/<testScenario>` folder. Here is the generated file folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202105120922-5c3x5` is current runId. For each run command it will generated unique runId.
+When run `run`, it automatically generate postman collection and postman env in `generated/<providerNamespace>/<apiScenarioFile>/<runId>/<apiScenario>` folder. Here is the generated file folder structure. The `collection.json` and `env.json` is generated postman collection file and environment file. `202105120922-5c3x5` is current runId. For each run command it will generated unique runId.
 
 ```
 generated
@@ -104,7 +105,7 @@ After you import postman collection, you will get such requests. Now you could d
 
 #### 5. manual update example value
 
-After debug with postman, you need to rewrite back all the updated values and run `oav run <test-scenario-file> -e <env.json>` again. The result should be successful.
+After debug with postman, you need to rewrite back all the updated values and run `oav run <api-scenario-file> -e <env.json>` again. The result should be successful.
 
 ## Feedback
 

diff --git a/...rio/how-to/testScenarioWithARMTemplate.md → ...ario/how-to/apiScenarioWithARMTemplate.md b/...rio/how-to/testScenarioWithARMTemplate.md → ...ario/how-to/apiScenarioWithARMTemplate.md
@@ -1,4 +1,4 @@
-# Test scenario integrate with armTemplate
+# API scenario integrate with armTemplate
 
 ## Background
 
@@ -13,7 +13,7 @@ Here is an example about `generate unique resource name for signalR service`
 
 #### Generate unique resource name
 
-We use `armTemplate output` to overwrite `resourceName` variable and following `createResource` step will use this variable. Below is generate unique name armTemplate. This armTemplate output `resourceName` variables, so test scenario following step will using the output variable.
+We use `armTemplate output` to overwrite `resourceName` variable and following `createResource` step will use this variable. Below is generate unique name armTemplate. This armTemplate output `resourceName` variables, so API scenario following step will using the output variable.
 
 ```json
 {
@@ -38,7 +38,7 @@ We use `armTemplate output` to overwrite `resourceName` variable and following `
 }
 ```
 
-After we have this armTemplate, we could define current test scenario file. We defined `resourceName` variable globally. `./generate_unique_string.json` is armTemplate.
+After we have this armTemplate, we could define current API scenario file. We defined `resourceName` variable globally. `./generate_unique_string.json` is armTemplate.
 
 `SignalR_CreateOrUpdate.json`
 
@@ -80,8 +80,9 @@ After we have this armTemplate, we could define current test scenario file. We d
 scope: ResourceGroup
 variables:
   resourceName: ""
-testScenarios:
-  - description: Microsoft.SignalRService/signalR CRUD
+scenarios:
+  - scenario: quickStart
+    description: Microsoft.SignalRService/signalR CRUD
     steps:
       - step: Generate_Unique_string
         armTemplateDeployment: ./generate_unique_string.json

diff --git a/...tion/test-scenario/how-to/armTemplate.png → ...ation/api-scenario/how-to/armTemplate.png b/...tion/test-scenario/how-to/armTemplate.png → ...ation/api-scenario/how-to/armTemplate.png
diff --git a/...test-scenario/how-to/folder-structure.png → .../api-scenario/how-to/folder-structure.png b/...test-scenario/how-to/folder-structure.png → .../api-scenario/how-to/folder-structure.png
diff --git a/.../test-scenario/how-to/genTestScenario.gif → ...n/api-scenario/how-to/genTestScenario.gif b/.../test-scenario/how-to/genTestScenario.gif → ...n/api-scenario/how-to/genTestScenario.gif
diff --git a/documentation/api-scenario/how-to/generateABasicApiScenario.md b/documentation/api-scenario/how-to/generateABasicApiScenario.md
@@ -0,0 +1,57 @@
+# Generate a basic API scenario file
+
+## Prerequisite
+
+We use `oav` tools to generate basic API scenario. `oav` analyze swagger file and use swagger example as API scenario steps. So first, you need to install the latest oav.
+
+## Introduction
+
+`oav` support rule based API scenario file generation. We use this command to generate API scenario file.
+
+`oav generate-static-api-scenario --readme <readme> --tag <tag> --rules <generated-rules>`
+
+- readme: swagger readme file.
+- tag: which tag to generate. oav will analyze swagger file under the tag and generate API scenario.
+- rules: Currently support two types. `resource-put-delete`, `operations-list`. Default: `resource-put-delete`
+  - `resource-put-delete`: generate resource put and delete API scenario.
+  - `operations-list`: generate operations list API scenario. `operations-list` is the simplest API which must be defined in swagger.
+
+Example:
+
+![](./genTestScenario.gif)
+
+This command will load and analyze swagger and generate a basic API scenario file (`resource-put-delete`).
+
+Result: the output contains two files
+
+- scenarios/signalR.yaml: The API scenario file.
+- readme.test.md: The entry for SDK test generation
+
+The generated API scenario file: The generated API scenario file contains two steps. Create signalR and delete it. It's a basic API scenario and developer can add more step based on the basic API scenario file.
+
+```
+scope: ResourceGroup
+testScenarios:
+  - description: Microsoft.SignalRService/signalR SignalR_CreateOrUpdate
+    steps:
+      - step: SignalR_CreateOrUpdate
+        exampleFile: ../examples/SignalR_CreateOrUpdate.json
+      - step: SignalR_Delete
+        exampleFile: ../examples/SignalR_Delete.json
+```
+
+If you pass rule option `operations-list`, you will get such API scenario file.
+
+```
+scope: ResourceGroup
+testScenarios:
+  - description: operationsList
+    steps:
+      - step: operationsList
+        exampleFile: ../examples/Operations_List.json
+
+```
+
+## Reference
+
+- [oav](https://github.com/Azure/oav/tree/develop)
diff --git a/...ario/how-to/import-postman-collection.png → ...ario/how-to/import-postman-collection.png b/...ario/how-to/import-postman-collection.png → ...ario/how-to/import-postman-collection.png
diff --git a/...rio/how-to/postman-collection-signalr.PNG → ...rio/how-to/postman-collection-signalr.PNG b/...rio/how-to/postman-collection-signalr.PNG → ...rio/how-to/postman-collection-signalr.PNG
diff --git a/...ation/test-scenario/how-to/runApiTest.gif → ...tation/api-scenario/how-to/runApiTest.gif b/...ation/test-scenario/how-to/runApiTest.gif → ...tation/api-scenario/how-to/runApiTest.gif
diff --git a/documentation/api-scenario/readme.md b/documentation/api-scenario/readme.md
@@ -0,0 +1,29 @@
+# API Scenario Documentation
+
+API Scenario is a YAML file defining RESTful API usage scenarios of your service with a series of API calls. API scenario can be used for service functional test, API quality validation and SDK/CLIs test generation.
+
+_**Caution**: This project is in early preview phase, hence breaking changes should be expected._
+
+## Features
+
+- Simple to use: Intuitive step definition based on Swagger examples and raw REST call.
+- ARM Template integration: Support creating external Azure resources with ARM Template and executing Azure Powershell or Azure CLI scripts with ARM Template deployment script.
+- Implementation independent: [oav](https://github.com/Azure/oav) is the default API scenario runner, and more runners will be supported, like SDKs in different languages.
+
+### Demo gif
+
+![demo](./how-to/runApiTest.gif)
+
+## Quick start
+
+- [Example: Write and run your first API scenario file](./how-to/QuickStart.md)
+- [Example: Generate a basic API scenario file](./how-to/generateABasicApiScenario.md)
+- [Example: use armTemplate to generate unique resourceName](./how-to/apiScenarioWithARMTemplate.md)
+- [API scenario file sample](../samplefiles/Microsoft.YourServiceName/stable/YYYY-MM-DD/scenarios/quickStart.yaml)
+
+## References
+
+- [API Scenario Definition Reference](./references/ApiScenarioDefinition.md)
+- [API Scenario Variable Definition Reference](./references/Variables.md)
+- [API Scenario Runner Reference](./references/Runner.md)
+- [API Scenario JSON Schema](./references/v1.1/schema.json)