Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix swagger definition. #26846

Merged
merged 11 commits into from
Dec 5, 2023
Merged

fix swagger definition. #26846

merged 11 commits into from
Dec 5, 2023

Conversation

wangbwn
Copy link
Member

@wangbwn wangbwn commented Nov 27, 2023

ARM (Control Plane) API Specification Update Pull Request

PR review workflow diagram

Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

diagram

Click here to see the details of Step 1

Breaking changes review (Diagram Step 1)

If the automation determines you have breaking changes, i.e. Step 1 from the diagram applies to you,
you must follow the breaking changes process.
IMPORTANT This applies even if:

  • The tool fails while it shouldn't, e.g. due to runtime exception, or incorrect detection of breaking changes.
  • You believe there is no need for you to request breaking change approval, for any reason.
    Such claims must be reviewed, and the process is the same.
Click here to see the details of Step 2

ARM API changes review (Diagram Step 2)

  • If this PR is in purview of ARM review then automation will add the ARMReview label.
  • If you want to force ARM review, add the label yourself.
  • Proceed according to the diagram at the top of this comment.
Click here to see the diagram footnotes

Diagram footnotes

[1] ARM review queue (for merge queues, see [4])
The PRs are processed by time opened, ascending. Your PR may show up on 2nd or later page.
If you addressed Step 1 from the diagram and your PR is not showing up in the queue, ensure the label ARMChangesRequested
is removed from your PR. This should cause the label WaitForARMFeedback to be added.
[2] https://aka.ms/azsdk/support/specreview-channel
[3] List of SDK breaking changes approvers in pinned Teams announcement
[4] public repo merge queue, private repo merge queue (for ARM review queue, [1])

If you need further help with anything, see Getting help section below.

Purpose of this PR

What's the purpose of this PR? Check all that apply. This is mandatory!

  • New API version. (If API spec is not defined in TypeSpec, the PR should have been generated using OpenAPI Hub).
  • Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
  • Update existing version to fix swagger quality issues in S360.
  • Other, please clarify:
    • edit this with your clarification

Due diligence checklist

To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

  • I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
  • I have reviewed following Resource Provider guidelines, including
    ARM resource provider contract and
    REST guidelines (estimated time: 4 hours).
    I understand this is required before I can proceed to the Diagram Step 2, "ARM API changes Review", for this PR.

Additional information

Viewing API changes

For convenient view of the API changes made by this PR, refer to the URLs provided in the table
in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures

If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the
Swagger-Suppression-Process
to get approval.

Getting help

  • For general PR approval workflow, see the diagram at the top of this comment.
  • For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
    and https://aka.ms/ci-fix.
  • If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run.
    This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
  • If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

Copy link

openapi-pipeline-app bot commented Nov 27, 2023

Next Steps to Merge

✔️ All automated merging requirements have been met! Refer to step 4 in the PR workflow diagram (even if your PR is for data plane, not ARM).

Copy link

Hi @wangbwn, this service has enabled Service API Toolset. All spec updates MUST be initiated from service ADO project repo, so that to guarantee it to be the source of truth., please review this pull request if it's intentional or reject it if it's not expected. Normally, all the specification or example changes should start with PR created in ADO.

  • You can refer to Service API Toolset Introduction for details.
  • Please contact service focal contacts or Service API Toolset Support Channel if need further help.
  • Copy link

    openapi-pipeline-app bot commented Nov 27, 2023

    Swagger Validation Report

    ️️✔️BreakingChange succeeded [Detail] [Expand]
    There are no breaking changes.
    compared swaggers (via Oad v0.10.4)] new version base version
    Dataset.json 2018-06-01(f85dddf) 2018-06-01(main)
    IntegrationRuntime.json 2018-06-01(f85dddf) 2018-06-01(main)
    LinkedService.json 2018-06-01(f85dddf) 2018-06-01(main)
    Pipeline.json 2018-06-01(f85dddf) 2018-06-01(main)
    ️️✔️Breaking Change(Cross-Version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️CredScan succeeded [Detail] [Expand]
    There is no credential detected.
    ️⚠️LintDiff: 0 Warnings warning [Detail]
    compared tags (via openapi-validator v2.1.6) new version base version
    package-2018-06 package-2018-06(f85dddf) package-2018-06(main)

    The following errors/warnings exist before current PR submission:

    Only 30 items are listed, please refer to log for more details.

    Rule Message
    ⚠️ AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
    Location: stable/2018-06-01/entityTypes/Dataset.json#L61
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L614
    ⚠️ SchemaDescriptionOrTitle Schema should have a description or title.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L633
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L667
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L725
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L759
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L853
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L891
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L934
    ⚠️ SchemaDescriptionOrTitle Schema should have a description or title.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L948
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/Dataset.json#L982
    ⚠️ LocationMustHaveXmsMutability Property location must have 'x-ms-mutability':['read', 'create'] extension defined.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L104
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L169
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L652
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1131
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1292
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1341
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1646
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1650
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1670
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/IntegrationRuntime.json#L1701
    ⚠️ AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L68
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L273
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L277
    ⚠️ SchemaDescriptionOrTitle Schema should have a description or title.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L3188
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L6479
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L6766
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L6786
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L7022
    ⚠️ EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
    Location: stable/2018-06-01/entityTypes/Pipeline.json#L7031
    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️️✔️SwaggerAPIView succeeded [Detail] [Expand]
    ️️✔️TypeSpecAPIView succeeded [Detail] [Expand]
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️️✔️PoliCheck succeeded [Detail] [Expand]
    Validation passed for PoliCheck.
    ️️✔️SpellCheck succeeded [Detail] [Expand]
    Validation passes for SpellCheck.
    ️️✔️Lint(RPaaS) succeeded [Detail] [Expand]
    Validation passes for Lint(RPaaS).
    ️️✔️PR Summary succeeded [Detail] [Expand]
    Validation passes for Summary.
    ️️✔️Automated merging requirements met succeeded [Detail] [Expand]
    Posted by Swagger Pipeline | How to fix these errors?

    Copy link

    openapi-pipeline-app bot commented Nov 27, 2023

    Swagger Generation Artifacts

    ️️✔️ApiDocPreview succeeded [Detail] [Expand]
     Please click here to preview with your @microsoft account. 
    ️️✔️SDK Breaking Change Tracking succeeded [Detail] [Expand]

    Breaking Changes Tracking




    ️❌ azure-sdk-for-net-track2 failed [Detail]
    • Failed [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	pwsh ./eng/scripts/Automation-Sdk-Init.ps1 ../azure-sdk-for-net_tmp/initInput.json ../azure-sdk-for-net_tmp/initOutput.json
      command	pwsh ./eng/scripts/Invoke-GenerateAndBuildV2.ps1 ../azure-sdk-for-net_tmp/generateInput.json ../azure-sdk-for-net_tmp/generateOutput.json
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[31;1mGeneratePackage: �[0m/mnt/vss/_work/1/s/azure-sdk-for-net/eng/scripts/automation/GenerateAndBuildLib.ps1:714
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1mLine |
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1m 714 | �[0m         �[36;1mGeneratePackage -projectFolder $projectFolder -sdkRootPath $s�[0m …
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[36;1m     | �[31;1m         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[31;1m�[36;1m     | �[31;1mFailed to build sdk. exit code: False
      cmderr	[Invoke-GenerateAndBuildV2.ps1] �[0m
    • Azure.ResourceManager.DataFactory [View full logs]  [Preview SDK Changes]
      info	[Changelog]
    ️⚠️ azure-sdk-for-python-track2 warning [Detail]
    • ⚠️Warning [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	sh scripts/automation_init.sh ../azure-sdk-for-python_tmp/initInput.json ../azure-sdk-for-python_tmp/initOutput.json
      cmderr	[automation_init.sh] WARNING: azure-devtools 1.2.1 does not provide the extra 'ci-tools'
      cmderr	[automation_init.sh] WARNING: azure-devtools 1.2.1 does not provide the extra 'ci-tools'
      cmderr	[automation_init.sh] WARNING: Skipping azure-nspkg as it is not installed.
      command	sh scripts/automation_generate.sh ../azure-sdk-for-python_tmp/generateInput.json ../azure-sdk-for-python_tmp/generateOutput.json
      cmderr	[automation_generate.sh] npm notice
      cmderr	[automation_generate.sh] npm notice New major version of npm available! 9.8.1 -> 10.2.4
      cmderr	[automation_generate.sh] npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.2.4>
      cmderr	[automation_generate.sh] npm notice Run `npm install -g npm@10.2.4` to update!
      cmderr	[automation_generate.sh] npm notice
    • ️✔️track2_azure-mgmt-datafactory [View full logs]  [Preview SDK Changes]
      info	[Changelog] ### Features Added
      info	[Changelog]
      info	[Changelog]   - Model WebActivity has a new parameter http_request_timeout
      info	[Changelog]   - Model WebActivity has a new parameter turn_off_async
    ️⚠️ azure-sdk-for-java warning [Detail]
    • ⚠️Warning [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	./eng/mgmt/automation/init.sh ../azure-sdk-for-java_tmp/initInput.json ../azure-sdk-for-java_tmp/initOutput.json
      cmderr	[init.sh] [notice] A new release of pip is available: 23.0.1 -> 23.3.1
      cmderr	[init.sh] [notice] To update, run: pip install --upgrade pip
      cmderr	[init.sh] [notice] A new release of pip is available: 23.0.1 -> 23.3.1
      cmderr	[init.sh] [notice] To update, run: pip install --upgrade pip
      command	./eng/mgmt/automation/generate.py ../azure-sdk-for-java_tmp/generateInput.json ../azure-sdk-for-java_tmp/generateOutput.json
    • ️✔️azure-resourcemanager-datafactory [View full logs]  [Preview SDK Changes]
    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	sh ./eng/scripts/automation_init.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
      command	generator automation-v2 ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json
    • ️✔️sdk/resourcemanager/datafactory/armdatafactory [View full logs]  [Preview SDK Changes]
      info	[Changelog] ### Features Added
      info	[Changelog]
      info	[Changelog] - New field `HTTPRequestTimeout`, `TurnOffAsync` in struct `WebActivityTypeProperties`
      info	[Changelog]
      info	[Changelog] Total 0 breaking change(s), 1 additive change(s).
    ️️✔️ azure-sdk-for-js succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	sh .scripts/automation_init.sh ../azure-sdk-for-js_tmp/initInput.json ../azure-sdk-for-js_tmp/initOutput.json
      warn	File azure-sdk-for-js_tmp/initOutput.json not found to read
      command	sh .scripts/automation_generate.sh ../azure-sdk-for-js_tmp/generateInput.json ../azure-sdk-for-js_tmp/generateOutput.json
    • ️✔️@azure/arm-datafactory [View full logs]  [Preview SDK Changes]
      info	[Changelog] **Features**
      info	[Changelog]
      info	[Changelog]   - Interface WebActivity has a new optional parameter httpRequestTimeout
      info	[Changelog]   - Interface WebActivity has a new optional parameter turnOffAsync
    ️⚠️ azure-resource-manager-schemas warning [Detail]
    • ⚠️Warning [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. Schema Automation 14.0.0
      command	.sdkauto/initScript.sh ../azure-resource-manager-schemas_tmp/initInput.json ../azure-resource-manager-schemas_tmp/initOutput.json
      cmderr	[initScript.sh] npm notice
      cmderr	[initScript.sh] npm notice New major version of npm available! 8.19.4 -> 10.2.4
      cmderr	[initScript.sh] npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.2.4>
      cmderr	[initScript.sh] npm notice Run `npm install -g npm@10.2.4` to update!
      cmderr	[initScript.sh] npm notice
      warn	File azure-resource-manager-schemas_tmp/initOutput.json not found to read
      command	.sdkauto/generateScript.sh ../azure-resource-manager-schemas_tmp/generateInput.json ../azure-resource-manager-schemas_tmp/generateOutput.json
    • ️✔️datafactory [View full logs]  [Preview Schema Changes]
    ️❌ azure-powershell failed [Detail]
    • Pipeline Framework Failed [Logs] Generate from 2880f57e2fc48204340b6f478ad743a1e8f2a11f. SDK Automation 14.0.0
      command	sh ./tools/SwaggerCI/init.sh ../azure-powershell_tmp/initInput.json ../azure-powershell_tmp/initOutput.json
      command	pwsh ./tools/SwaggerCI/psci.ps1 ../azure-powershell_tmp/generateInput.json ../azure-powershell_tmp/generateOutput.json
      SSL error: syscall failure: Broken pipe
      Error: SSL error: syscall failure: Broken pipe
    • ⚠️Az.datafactory.DefaultTag [View full logs
      error	Fatal error: SSL error: syscall failure: Broken pipe
      error	The following packages are still pending:
      error		Az.datafactory.DefaultTag
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-workflow-bot openapi-workflow-bot bot added the DoNotMerge <valid label in PR review process> use to hold merge after approval label Nov 27, 2023
    Copy link

    openapi-pipeline-app bot commented Nov 27, 2023

    Generated ApiView

    Language Package Name ApiView Link
    Go sdk/resourcemanager/datafactory/armdatafactory https://apiview.dev/Assemblies/Review/0c4899702ed542dabec855f31d989b3f
    Java azure-resourcemanager-datafactory https://apiview.dev/Assemblies/Review/0fe228f350b5439ab613799a5198d476
    JavaScript @azure/arm-datafactory https://apiview.dev/Assemblies/Review/08e4cdfd850b47459868422e77d4d630
    .Net Azure.ResourceManager.DataFactory Create ApiView timeout. Package is too large and we cannot create ApiView for it.
    Swagger Microsoft.DataFactory https://apiview.dev/Assemblies/Review/5e3133c32add49ea8245694575c3957f

    @Jingshu918 Jingshu918 removed the DoNotMerge <valid label in PR review process> use to hold merge after approval label Nov 29, 2023
    @ruowan
    Copy link
    Member

    ruowan commented Nov 29, 2023

    The Snowflake linkedService requires an update to the connectionString property to include 'x-ms-format'. Could you please confirm this? Additionally, it seems that the PR may not cover all instances of incorrect 'x-ms-format' properties listed in excel. Could you verify?

    @wangbwn
    Copy link
    Member Author

    wangbwn commented Nov 29, 2023

    The Snowflake linkedService requires an update to the connectionString property to include 'x-ms-format'. Could you please confirm this? Additionally, it seems that the PR may not cover all instances of incorrect 'x-ms-format' properties listed in excel. Could you verify?

    From the adf it should look like dfe-string.

    @sjanamma sjanamma added the ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review label Dec 2, 2023
    @openapi-workflow-bot openapi-workflow-bot bot removed the WaitForARMFeedback <valid label in PR review process> add this label when ARM review is required label Dec 2, 2023
    @ruowan
    Copy link
    Member

    ruowan commented Dec 5, 2023

    /azp run

    Copy link

    Azure Pipelines successfully started running 3 pipeline(s).

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    ARMReview ARMSignedOff <valid label in PR review process>add this label when ARM approve updates after review data-plane ReadyForApiTest <valid label in PR review process>add this label when swagger and service APIs are ready for test resource-manager ShiftLeftViolation
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    9 participants