Helix Client Testability and Stability Improvements

[missymessa]

We want to improve the testability of the Helix Scripts. I included issue #7107 in this epic that was the result of a discussion with SMEs on the team regarding the lack of ability to test and dependency inject into our scripts.

A suggestion for improving the testability of the Helix Scripts would be to rewrite them using .NET Core. Since .NET Core is more stable than it was when the Helix Script were originally written, this is preferable to refactoring the existing Python scripts.

Recently Triaged Issues

All issues in this section should be triaged by the v-team into one of their business objectives or features.

• https://github.com/dotnet/core-eng/issues/12099
• Plan for Improved Testability of Helix Python Scripts #7107
• https://github.com/dotnet/core-eng/issues/12916
• Consider moving arcade usages of ADO "attachments" APIs to new testlogstoreendpoint #7319
• https://github.com/dotnet/core-eng/issues/13361

[premun]

It would be super cool to have .NET Helix client but how are we addressing the fact we need to bootstrap new platforms where there is no .NET available?

[tkapin]

I support rewriting the Helix client in C#, primarily because of difficulties with deployment of Python solutions (venvs, pip dependency in runtime, etc.). As for supporting platforms where .NET Core is not running yet, I'm not that much worried about *Nix systems as mono covers really broad range of platforms and architectures already (see https://www.mono-project.com/docs/about-mono/supported-platforms/) as we could have leveraged that to run the new client on these platforms if we decided to support them.

[missymessa]

Following up on this, while adding functionality to allow retries of tests, @ChadNedzlek attempted to refactor the existing Python scripts to have dependency injection support. Unfortunately, the attempt was scrapped as it was becoming unwieldly (among other issues). Based on that, I've also been in favor of rewriting these scripts using .NET Core.

[dougbu]

This epic came up today in relation to dotnet/dnceng#427 and #13186. Python on older platforms (those that are out of support from the provider's perspective but in support from .NET's perspective /fyi @richlander) is not currently looking like a sustainable approach. If we can't switch gears and migrate to CloudTest, we likely need to invest in porting the Helix client.

Note this is not an immediate problem because (a) we haven't looked enough at workarounds for current Python issues and (b) we haven't given ourselves enough time to finish the CloudTest exploration. Put another way, CloudTest is Plan A and this issue is Plan C or so.