[DataFactory]Add support for running staging on multiple sources in parallel #18956

kesharma · 2022-05-09T01:50:05Z

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Add a changelog entry for this PR by answering the following questions:

What's the purpose of the update?
- new service onboarding
- new API version
- update existing version for new feature
- update existing version to fix swagger quality issue in s360
- Other, please clarify
When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
If updating an existing version, please select the specific language SDKs and CLIs that must be refreshed after the swagger is published.
- SDK of .NET (need service team to ensure code readiness)
- SDK of Python
- SDK of Java
- SDK of Js
- SDK of Go
- PowerShell
- CLI
- Terraform
- No refresh required for updates in this PR

Contribution checklist:

I commit to follow the Breaking Change Policy of "no breaking changes"
I have reviewed the documentation for the workflow.
Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

Applicability: ⚠️

If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

Change to data plane APIs

Adding new properties

All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

Check this box if any of the following apply to the PR so that label "WaitForARMFeedback" will be added automatically to begin ARM API Review. Failure to comply may result in delays to the manifest.
- Adding a new service
- Adding new API(s)
- Adding a new API version
  -[ ] To review changes efficiently, ensure you are using OpenAPIHub to initialize the PR for adding a new version. More details, refer to the wiki.
Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.
If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If any of the following scenarios apply to the PR, request approval from the Breaking Change Review Board as defined in the Breaking Change Policy.

Removing API(s) in a stable version
Removing properties in a stable version
Removing API version(s) in a stable version
Updating API in a stable or public preview version with Breaking Change Validation errors
Updating API(s) in public preview over 1 year (refer to Retirement of Previews)

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

openapi-workflow-bot · 2022-05-09T01:50:08Z

Hi, @kesharma Thanks for your PR. I am workflow bot for review process. Here are some small tips.

Please ensure to do self-check against checklists in first PR comment.

PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.

For specs comparison cross API versions, Use API Specs Comparison Report Generator

If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

Any feedback about review process or workflow bot, pls contact swagger and tools team. vscswagger@microsoft.com

openapi-pipeline-app · 2022-05-09T01:50:09Z

Swagger Validation Report

️❌BreakingChange: 2 Errors, 0 Warnings failed [Detail]

Compared Swaggers (Based on Oad v0.9.5)
- current:2018-06-01/entityTypes/Pipeline.json compared with base:2018-06-01/entityTypes/Pipeline.json

Rule	Message
`❌` 1045 - AddedOptionalProperty	The new version has a new optional property 'sourceStagingConcurrency' that was not found in the old version. New: stable/2018-06-01/entityTypes/Pipeline.json#L7386:7 Old: stable/2018-06-01/entityTypes/Pipeline.json#L7386:7
`❌` 1045 - AddedOptionalProperty	The new version has a new optional property 'sourceStagingConcurrency' that was not found in the old version. New: stable/2018-06-01/entityTypes/Pipeline.json#L7442:7 Old: stable/2018-06-01/entityTypes/Pipeline.json#L7438:7

️⚠️LintDiff: 0 Warnings warning [Detail]

Linted configuring files (Based on source branch, openapi-validator v1.13.0 , classic-openapi-validator v1.2.4 )
- datafactory/resource-manager/readme.md tag:package-2018-06
Linted configuring files (Based on target branch, openapi-validator v1.13.0 , classic-openapi-validator v1.2.4 )
- datafactory/resource-manager/readme.md tag:package-2018-06

The following errors/warnings exist before current PR submission:

Only 30 items are listed, please refer to log for more details.

Rule	Message
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L46
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L46
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L238
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L238
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5263
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5263
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L6114
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L6114
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L6534
`❌` R4013 - IntegerTypeMustHaveFormat	The integer type does not have a format, please add it. Location: stable/2018-06-01/entityTypes/Pipeline.json#L6534
`❌` R4037 - MissingTypeObject	The schema 'ControlActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L197
`❌` R4037 - MissingTypeObject	The schema 'ControlActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L197
`❌` R4037 - MissingTypeObject	The schema 'ExecutionActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L207
`❌` R4037 - MissingTypeObject	The schema 'ExecutionActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L207
`❌` R4037 - MissingTypeObject	The schema 'CopyActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L1536
`❌` R4037 - MissingTypeObject	The schema 'CopyActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L1536
`❌` R4037 - MissingTypeObject	The schema 'CopyActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L1575
`❌` R4037 - MissingTypeObject	The schema 'CopyActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L1575
`❌` R4037 - MissingTypeObject	The schema 'HDInsightHiveActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5193
`❌` R4037 - MissingTypeObject	The schema 'HDInsightHiveActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5193
`❌` R4037 - MissingTypeObject	The schema 'HDInsightHiveActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5212
`❌` R4037 - MissingTypeObject	The schema 'HDInsightHiveActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5212
`❌` R4037 - MissingTypeObject	The schema 'HDInsightPigActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5282
`❌` R4037 - MissingTypeObject	The schema 'HDInsightPigActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5282
`❌` R4037 - MissingTypeObject	The schema 'HDInsightPigActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5301
`❌` R4037 - MissingTypeObject	The schema 'HDInsightPigActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5301
`❌` R4037 - MissingTypeObject	The schema 'HDInsightMapReduceActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5340
`❌` R4037 - MissingTypeObject	The schema 'HDInsightMapReduceActivity' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5340
`❌` R4037 - MissingTypeObject	The schema 'HDInsightMapReduceActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5359
`❌` R4037 - MissingTypeObject	The schema 'HDInsightMapReduceActivityTypeProperties' is considered an object but without a 'type:object', please add the missing 'type:object'. Location: stable/2018-06-01/entityTypes/Pipeline.json#L5359

️️✔️Avocado succeeded [Detail] [Expand]

Validation passes for Avocado.

️️✔️ApiReadinessCheck succeeded [Detail] [Expand]

️️✔️ModelValidation succeeded [Detail] [Expand]

Validation passes for ModelValidation.

️️✔️SemanticValidation succeeded [Detail] [Expand]

Validation passes for SemanticValidation.

️️✔️Cross-Version Breaking Changes succeeded [Detail] [Expand]

There are no breaking changes.

️️✔️CredScan succeeded [Detail] [Expand]

There is no credential detected.

️️✔️PoliCheck succeeded [Detail] [Expand]

Validation passed for PoliCheck.

️️✔️SDK Track2 Validation succeeded [Detail] [Expand]

Validation passes for SDKTrack2Validation

The following tags are being changed in this PR

datafactory/resource-manager/readme.md#package-2018-06

️️✔️PrettierCheck succeeded [Detail] [Expand]

Validation passes for PrettierCheck.

️️✔️SpellCheck succeeded [Detail] [Expand]

Validation passes for SpellCheck.

️️✔️Lint(RPaaS) succeeded [Detail] [Expand]

Validation passes for Lint(RPaaS).

_{Posted by Swagger Pipeline | How to fix these errors?}

openapi-pipeline-app · 2022-05-09T01:50:10Z

Swagger Generation Artifacts

️️✔️ApiDocPreview succeeded [Detail] [Expand]

 Please click here to preview with your @microsoft account.

️❌SDK Breaking Change Tracking failed [Detail]

Breaking Changes Tracking

️✔️azure-sdk-for-go-track2 - armdatafactory - 0.6.0
️✔️azure-sdk-for-python-track2 - track2_azure-mgmt-datafactory - 2.4.0
️✔️azure-sdk-for-go - datafactory/mgmt/2018-06-01/datafactory - v64.0.0
❌azure-sdk-for-go - preview/datafactory/mgmt/2017-09-01-preview/datafactory - v64.0.0

️❌ azure-sdk-for-net failed [Detail]

❌Failed [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

warn	Skip initScript due to not configured
command	sudo apt-get install -y dotnet-sdk-6.0
command	autorest --version=2.0.4421 --csharp --reflect-api-versions --license-header=MICROSOFT_MIT_NO_VERSION --use=@microsoft.azure/autorest.csharp@2.3.82 --csharp-sdks-folder=/mnt/vss/_work/1/s/azure-sdk-for-net/sdk ../azure-rest-api-specs/specification/datafactory/resource-manager/readme.md
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] realpath(): Permission denied
cmderr	[Autorest] No usable version of the libssl was found
cmderr	[Autorest] FATAL: csharp/simplifier - FAILED
cmderr	[Autorest] FATAL: Error: [Exception] AutoRest extension '@microsoft.azure/autorest.csharp' terminated.
cmderr	[Autorest] Process() cancelled due to exception : [Exception] AutoRest extension '@microsoft.azure/autorest.csharp' terminated.
error	Script return with result [failed] code [1] signal [null] cwd [azure-sdk-for-net]: autorest
warn	Skip package processing as generation is failed

️⚠️ azure-sdk-for-python-track2 warning [Detail]

⚠️Warning [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	sh scripts/automation_init.sh ../azure-sdk-for-python_tmp/initInput.json ../azure-sdk-for-python_tmp/initOutput.json
cmderr	[automation_init.sh] WARNING: Skipping azure-nspkg as it is not installed.
command	sh scripts/automation_generate.sh ../azure-sdk-for-python_tmp/generateInput.json ../azure-sdk-for-python_tmp/generateOutput.json

️✔️track2_azure-mgmt-datafactory [View full logs] [Release SDK Changes]

info	[Changelog] **Features**
info	[Changelog]
info	[Changelog]   - Model DataFlowSink has a new parameter rejected_data_linked_service
info	[Changelog]   - Model ExecuteDataFlowActivity has a new parameter source_staging_concurrency
info	[Changelog]   - Model ExecuteDataFlowActivityTypeProperties has a new parameter source_staging_concurrency
info	[Changelog]   - Model ExecutePowerQueryActivityTypeProperties has a new parameter source_staging_concurrency
info	[Changelog]   - Model ExecuteWranglingDataflowActivity has a new parameter source_staging_concurrency
info	[Changelog]   - Model Factory has a new parameter purview_configuration
info	[Changelog]   - Model PowerQuerySink has a new parameter rejected_data_linked_service
info	[Changelog]   - Model PrivateLinkConnectionApprovalRequest has a new parameter private_endpoint

️⚠️ azure-sdk-for-java warning [Detail]

⚠️Warning [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	./eng/mgmt/automation/init.sh ../azure-sdk-for-java_tmp/initInput.json ../azure-sdk-for-java_tmp/initOutput.json
command	./eng/mgmt/automation/generate.py ../azure-sdk-for-java_tmp/generateInput.json ../azure-sdk-for-java_tmp/generateOutput.json
cmderr	[generate.py] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
cmderr	[generate.py] SLF4J: Defaulting to no-operation (NOP) logger implementation
cmderr	[generate.py] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
cmderr	[generate.py] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
cmderr	[generate.py] SLF4J: Defaulting to no-operation (NOP) logger implementation
cmderr	[generate.py] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
cmderr	[generate.py] [ForkJoinPool-1-worker-3] INFO com.azure.core.test.implementation.TestingHelpers - Environment variable 'AZURE_TEST_MODE' has not been set yet. Using 'Playback' mode.

️✔️azure-resourcemanager-datafactory [View full logs] [Release SDK Changes]

️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]

️✔️Succeeded [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	sh ./eng/initScript.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
command	generator automation ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json

️✔️datafactory/mgmt/2018-06-01/datafactory [View full logs] [Release SDK Changes]

info	[Changelog] ### Features Added
info	[Changelog]
info	[Changelog] - New struct `PurviewConfiguration`
info	[Changelog] - New field `SourceStagingConcurrency` in struct `ExecutePowerQueryActivityTypeProperties`
info	[Changelog] - New field `SourceStagingConcurrency` in struct `ExecuteDataFlowActivityTypeProperties`
info	[Changelog] - New field `RejectedDataLinkedService` in struct `PowerQuerySink`
info	[Changelog] - New field `RejectedDataLinkedService` in struct `DataFlowSink`
info	[Changelog] - New field `PurviewConfiguration` in struct `FactoryProperties`
info	[Changelog]
info	[Changelog] Total 0 breaking change(s), 7 additive change(s).

️✔️preview/datafactory/mgmt/2017-09-01-preview/datafactory [View full logs]

Only show 60 items here, please refer to log for details.

info	[Changelog] ### Features Added
info	[Changelog]
info	[Changelog] - New const `Value`
info	[Changelog] - New const `ONE`
info	[Changelog] - New const `QUORUM`
info	[Changelog] - New const `IntegrationRuntimeStateStarting`
info	[Changelog] - New const `ALL`
info	[Changelog] - New const `QueryAll`
info	[Changelog] - New const `THREE`
info	[Changelog] - New const `Date`
info	[Changelog] - New const `SERIAL`
info	[Changelog] - New const `Ifd`
info	[Changelog] - New const `Optimal`
info	[Changelog] - New const `LOCALONE`
info	[Changelog] - New const `EACHQUORUM`
info	[Changelog] - New const `Merge`
info	[Changelog] - New const `Available`
info	[Changelog] - New const `Insert`
info	[Changelog] - New const `TWO`
info	[Changelog] - New const `String`
info	[Changelog] - New const `SapCloudForCustomerSinkWriteBehaviorInsert`
info	[Changelog] - New const `Decimal`
info	[Changelog] - New const `IntegrationRuntimeStateStopped`
info	[Changelog] - New const `Unavailable`
info	[Changelog] - New const `GUID`
info	[Changelog] - New const `IntegrationRuntimeStateOnline`
info	[Changelog] - New const `OnPremisesWithIfd`
info	[Changelog] - New const `Fastest`
info	[Changelog] - New const `IntegrationRuntimeStateStarted`
info	[Changelog] - New const `SapCloudForCustomerSinkWriteBehaviorUpdate`
info	[Changelog] - New const `LOCALQUORUM`
info	[Changelog] - New const `IntegrationRuntimeStateNeedRegistration`
info	[Changelog] - New const `IntegrationRuntimeStateStopping`
info	[Changelog] - New const `Office365`
info	[Changelog] - New const `IntegrationRuntimeStateLimited`
info	[Changelog] - New const `Disabled`
info	[Changelog] - New const `IntegrationRuntimeStateInitial`
info	[Changelog] - New const `IntegrationRuntimeStateOffline`
info	[Changelog] - New const `Query`
info	[Changelog] - New const `Upload`
info	[Changelog] - New const `Boolean`
info	[Changelog] - New const `Recycling`
info	[Changelog] - New const `Upsert`
info	[Changelog] - New const `LOCALSERIAL`
info	[Changelog] - New const `Int`
info	[Changelog] - New const `Int64`
info	[Changelog] - New const `Percentage`
info	[Changelog] - New function `PossibleSalesforceSourceReadBehaviorValues() []SalesforceSourceReadBehavior`
info	[Changelog] - New function `PossiblePolybaseSettingsRejectTypeValues() []PolybaseSettingsRejectType`
info	[Changelog] - New function `PossibleSapCloudForCustomerSinkWriteBehaviorValues() []SapCloudForCustomerSinkWriteBehavior`
info	[Changelog] - New function `PossibleStoredProcedureParameterTypeValues() []StoredProcedureParameterType`
info	[Changelog] - New function `PossibleSalesforceSinkWriteBehaviorValues() []SalesforceSinkWriteBehavior`
info	[Changelog] - New function `PossibleAzureSearchIndexWriteBehaviorTypeValues() []AzureSearchIndexWriteBehaviorType`
info	[Changelog] - New function `PossibleDatasetCompressionLevelValues() []DatasetCompressionLevel`
info	[Changelog] - New function `PossibleCassandraSourceReadConsistencyLevelsValues() []CassandraSourceReadConsistencyLevels`
info	[Changelog] - New function `PossibleDynamicsDeploymentTypeValues() []DynamicsDeploymentType`
info	[Changelog] - New function `PossibleDynamicsAuthenticationTypeValues() []DynamicsAuthenticationType`
info	[Changelog] - New field `DisableCertValidation` in struct `WebActivityTypeProperties`
info	[Changelog]
info	[Changelog] Total 28 breaking change(s), 56 additive change(s).

️️✔️ azure-sdk-for-go-track2 succeeded [Detail] [Expand]

️✔️Succeeded [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	sh ./eng/scripts/automation_init.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
command	generator automation-v2 ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json

️✔️armdatafactory [View full logs] [Release SDK Changes]

info	[Changelog] ### Features Added
info	[Changelog]
info	[Changelog] - New struct `PrivateEndpoint`
info	[Changelog] - New struct `PurviewConfiguration`
info	[Changelog] - New field `RejectedDataLinkedService` in struct `PowerQuerySink`
info	[Changelog] - New field `PurviewConfiguration` in struct `FactoryProperties`
info	[Changelog] - New field `PrivateEndpoint` in struct `PrivateLinkConnectionApprovalRequest`
info	[Changelog] - New field `RejectedDataLinkedService` in struct `DataFlowSink`
info	[Changelog] - New field `SourceStagingConcurrency` in struct `ExecutePowerQueryActivityTypeProperties`
info	[Changelog] - New field `SourceStagingConcurrency` in struct `ExecuteDataFlowActivityTypeProperties`
info	[Changelog]
info	[Changelog] Total 0 breaking change(s), 10 additive change(s).

️⚠️ azure-sdk-for-js warning [Detail]

⚠️Warning [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	sh .scripts/automation_init.sh ../azure-sdk-for-js_tmp/initInput.json ../azure-sdk-for-js_tmp/initOutput.json
cmderr	[automation_init.sh]  deprecated read-package-tree@5.1.6: The functionality that this package provided is now in @npmcli/arborist
cmderr	[automation_init.sh]  uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
cmderr	[automation_init.sh] npm WARN deprecated @opentelemetry/types@0.2.0: Package renamed to @opentelemetry/api, see https://github.com/open-telemetry/opentelemetry-js
cmderr	[automation_init.sh]  deprecated @azure/storage-blob@10.5.0: This version has been deprecated, please upgrade to the version tagged as latest
cmderr	[automation_init.sh] npm WARN @octokit/plugin-request-log@1.0.4 requires a peer of @octokit/core@>=3 but none is installed. You must install peer dependencies yourself.
warn	File azure-sdk-for-js_tmp/initOutput.json not found to read
command	sh .scripts/automation_generate.sh ../azure-sdk-for-js_tmp/generateInput.json ../azure-sdk-for-js_tmp/generateOutput.json

️✔️@azure/arm-datafactory [View full logs] [Release SDK Changes]

azure-arm-datafactory-10.5.0.tgz

info	[Changelog] **Features**
info	[Changelog]
info	[Changelog]   - Added Interface PrivateEndpoint
info	[Changelog]   - Added Interface PurviewConfiguration
info	[Changelog]   - Interface ExecuteDataFlowActivityTypeProperties has a new optional parameter sourceStagingConcurrency
info	[Changelog]   - Interface PrivateLinkConnectionApprovalRequest has a new optional parameter privateEndpoint
info	[Changelog]   - Type Alias DataFlowSink has a new parameter rejectedDataLinkedService
info	[Changelog]   - Type Alias ExecuteDataFlowActivity has a new parameter sourceStagingConcurrency
info	[Changelog]   - Type Alias ExecuteWranglingDataflowActivity has a new parameter sourceStagingConcurrency
info	[Changelog]   - Type Alias Factory has a new parameter purviewConfiguration
error	breakingChangeTracking is enabled, but version or changelogItem is not found in output.

️⚠️ azure-resource-manager-schemas warning [Detail]

⚠️Warning [Logs]Release - Generate from 0ceddfe. Schema Automation 14.0.0

command	.sdkauto/initScript.sh ../azure-resource-manager-schemas_tmp/initInput.json ../azure-resource-manager-schemas_tmp/initOutput.json
cmderr	[initScript.sh] WARN old lockfile
cmderr	[initScript.sh] npm WARN old lockfile The package-lock.json file was created with an old version of npm,
cmderr	[initScript.sh] npm WARN old lockfile so supplemental metadata must be fetched from the registry.
cmderr	[initScript.sh] npm WARN old lockfile
cmderr	[initScript.sh] npm WARN old lockfile This is a one-time fix-up, please be patient...
cmderr	[initScript.sh] npm WARN old lockfile
warn	File azure-resource-manager-schemas_tmp/initOutput.json not found to read
command	.sdkauto/generateScript.sh ../azure-resource-manager-schemas_tmp/generateInput.json ../azure-resource-manager-schemas_tmp/generateOutput.json

️✔️datafactory [View full logs] [Release Schema Changes]

️❌ azure-powershell failed [Detail]

❌Failed [Logs]Release - Generate from 0ceddfe. SDK Automation 14.0.0

command	sh ./tools/SwaggerCI/init.sh ../azure-powershell_tmp/initInput.json ../azure-powershell_tmp/initOutput.json
command	pwsh ./tools/SwaggerCI/psci.ps1 ../azure-powershell_tmp/generateInput.json ../azure-powershell_tmp/generateOutput.json

❌Az.datafactory [View full logs] [Release SDK Changes]

_{Posted by Swagger Pipeline | How to fix these errors?}

leni-msft · 2022-05-09T08:37:39Z

/azp run

azure-pipelines · 2022-05-09T08:37:49Z

Azure Pipelines successfully started running 1 pipeline(s).

openapi-workflow-bot · 2022-05-09T08:40:58Z

NewApiVersionRequired reason:

A service’s API is a contract with customers and is represented by using the api-version query parameter. Changes such as adding an optional property to a request/response or introducing a new operation is a change to the service’s contract and therefore requires a new api-version value. This is critically important for documentation, client libraries, and customer support.

EXAMPLE: if a customer calls a service in the public cloud using api-version=2020-07-27, the new property or operation may exist but if they call the service in a government cloud, air-gapped cloud, or Azure Stack Hub cloud using the same api-version, the property or operation may not exist. Because there is no clear relationship between the service api-version and the new property/operation, customers can’t trust the documentation and Azure customer have difficulty helping customers diagnose issues. In addition, each client library version documents the service version it supports. When an optional property or new operation is added to a service and its Swagger, new client libraries must be produced to expose this functionality to customers. Without updating the api-version, it is unclear to customers which version of a client library supports these new features.

leni-msft · 2022-05-09T09:30:15Z

bc approved in #18825

leni-msft · 2022-05-09T09:31:11Z

@tadelesh @ArcturusZhang please help check the go sdk generation failure.

ArcturusZhang · 2022-05-11T05:40:49Z

/azp run

azure-pipelines · 2022-05-11T05:40:58Z

Azure Pipelines successfully started running 1 pipeline(s).

…arallel (Azure#18956) * Add support for running staging on multiple sources in parallel * update

keshavsharma2024 added 2 commits April 26, 2022 08:48

Add support for running staging on multiple sources in parallel

ebc3dad

update

3b47848

kesharma requested review from Frey-Wang, zhangyd2015 and davidzhaoyue as code owners May 9, 2022 01:50

openapi-assignment-bot bot assigned leni-msft May 9, 2022

leni-msft approved these changes May 9, 2022

View reviewed changes

openapi-pipeline-app bot added the NewApiVersionRequired label May 9, 2022

leni-msft added the Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 label May 9, 2022

leni-msft approved these changes May 12, 2022

View reviewed changes

leni-msft merged commit 0ceddfe into Azure:main May 12, 2022

FredericHeem pushed a commit to grucloud/azure-rest-api-specs that referenced this pull request May 16, 2022

[DataFactory]Add support for running staging on multiple sources in p…

732d6d2

…arallel (Azure#18956) * Add support for running staging on multiple sources in parallel * update

Jingshu923 mentioned this pull request May 23, 2022

[DataFactory]Added new features into 6.2.0 Azure/azure-sdk-for-net#28916

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DataFactory]Add support for running staging on multiple sources in parallel #18956

[DataFactory]Add support for running staging on multiple sources in parallel #18956

kesharma commented May 9, 2022

openapi-workflow-bot bot commented May 9, 2022

openapi-pipeline-app bot commented May 9, 2022 •

edited

Loading

openapi-pipeline-app bot commented May 9, 2022 •

edited

Loading

Breaking Changes Tracking

leni-msft commented May 9, 2022

azure-pipelines bot commented May 9, 2022

openapi-workflow-bot bot commented May 9, 2022

leni-msft commented May 9, 2022

leni-msft commented May 9, 2022

ArcturusZhang commented May 11, 2022

azure-pipelines bot commented May 11, 2022

[DataFactory]Add support for running staging on multiple sources in parallel #18956

[DataFactory]Add support for running staging on multiple sources in parallel #18956

Conversation

kesharma commented May 9, 2022

Changelog

Contribution checklist:

ARM API Review Checklist

Breaking Change Review Checklist

openapi-workflow-bot bot commented May 9, 2022

openapi-pipeline-app bot commented May 9, 2022 • edited Loading

Swagger Validation Report

openapi-pipeline-app bot commented May 9, 2022 • edited Loading

Swagger Generation Artifacts

Breaking Changes Tracking

leni-msft commented May 9, 2022

azure-pipelines bot commented May 9, 2022

openapi-workflow-bot bot commented May 9, 2022

leni-msft commented May 9, 2022

leni-msft commented May 9, 2022

ArcturusZhang commented May 11, 2022

azure-pipelines bot commented May 11, 2022

openapi-pipeline-app bot commented May 9, 2022 •

edited

Loading

openapi-pipeline-app bot commented May 9, 2022 •

edited

Loading