Skip to content

Commit

Permalink
Cleanup PlatformSpecific/SkipOnMono attributes that skip a platform (#…
Browse files Browse the repository at this point in the history
…50907)

We have a bunch of test assemblies that don't make sense on some platforms, e.g. Browser.
Right now we're skipping them via `[SkipOnMono("reason", TestPlatforms.Browser)]` but there's nothing that inherently ties this to Mono other than the current implementation.

The more generic `SkipOnPlatform` attribute can be used instead.
We can also use it in places where we do `[PlatformSpecific(~TestPlatforms....)]` to avoid the double inversion.
  • Loading branch information
akoeplinger committed Apr 9, 2021
1 parent 6902c5a commit bc9f00c
Show file tree
Hide file tree
Showing 198 changed files with 372 additions and 348 deletions.
20 changes: 19 additions & 1 deletion docs/workflow/testing/libraries/filtering-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,24 @@ This attribute returns the 'failing' category, which is disabled by default.
```
Use this attribute over test methods to skip failing tests only on the specific platforms and the specific target frameworks.

#### SkipOnPlatformAttribute
This attribute is intended to disable a test permanently on a platform where an API is not available or there is an intentional difference in behavior in between the tested platform and the skipped platform.

This attribute can be applied either to a test assembly/class (will disable all the tests in that assembly/class) or to a test method. It allows multiple usages on the same member.

```cs
[SkipOnPlatform(TestPlatforms platforms, string reason)]
```

Use this attribute over test methods to skip tests only on the specific target platforms. The reason parameter doesn't affect the traits but we rather always use it so that when we see this attribute we know why it is being skipped on that platform.

If it needs to be skipped in multiple platforms and the reasons are different please use two attributes on the same test so that you can specify different reasons for each platform.

When you add the attribute on the whole test assembly it's a good idea to also add `<IgnoreForCI Condition="'$(TargetOS)' == '...'">true</IgnoreForCI>` to the test .csproj.
That allows the CI build to skip sending this test assembly to Helix completely since it'd run zero tests anyway.

**Currently these are the [Test Platforms](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.XUnitExtensions/src/TestPlatforms.cs) that we support through our test execution infrastructure**

#### SkipOnTargetFrameworkAttribute
This attribute is intended to disable a test permanently on a framework where an API is not available or there is an intentional difference in behavior in between the tested framework and the skipped framework.

Expand All @@ -80,7 +98,7 @@ Use this attribute over test methods to skip tests only on the specific target f

If it needs to be skipped in multiple frameworks and the reasons are different please use two attributes on the same test so that you can specify different reasons for each framework.

**Currently this are the [Framework Monikers](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.XUnitExtensions/src/TargetFrameworkMonikers.cs#L23-L26) that we support through our test execution infrastructure**
**Currently these are the [Framework Monikers](https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.XUnitExtensions/src/TargetFrameworkMonikers.cs#L23-L26) that we support through our test execution infrastructure**

#### ConditionalFactAttribute
Use this attribute to run the test only when a condition is `true`. This attribute is used when `ActiveIssueAttribute` or `SkipOnTargetFrameworkAttribute` are not flexible enough due to needing to run a custom logic at test time. This test behaves as a `[Fact]` test that has no test data passed in as a parameter.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
namespace System.IO.Tests
{
/// <summary>Base class providing tests for any Stream-derived type.</summary>
[PlatformSpecific(~TestPlatforms.Browser)] // lots of operations aren't supported on browser
[SkipOnPlatform(TestPlatforms.Browser, "lots of operations aren't supported on browser")]
public abstract class StreamConformanceTests : FileCleanupTestBase
{
/// <summary>Gets the name of the byte[] argument to Read/Write methods.</summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ namespace System.Security.Cryptography.Encryption.Aes.Tests
{
using Aes = System.Security.Cryptography.Aes;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class AesCipherTests
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ namespace System.Security.Cryptography.Encryption.Aes.Tests
{
using Aes = System.Security.Cryptography.Aes;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public class AesContractTests
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ namespace System.Security.Cryptography.Encryption.Aes.Tests
{
using Aes = System.Security.Cryptography.Aes;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class AesCornerTests
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ namespace System.Security.Cryptography.Encryption.Aes.Tests
{
using Aes = System.Security.Cryptography.Aes;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public class AesModeTests
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ namespace System.Security.Cryptography.Encryption.Aes.Tests
{
using Aes = System.Security.Cryptography.Aes;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DecryptorReusabilty
{
// See https://github.com/dotnet/runtime/issues/21354 for details
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

namespace System.Security.Cryptography.Encryption.Des.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DesCipherTests
{
// These are the expected output of many decryptions. Changing these values requires re-generating test input.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

namespace System.Security.Cryptography.Encryption.Des.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DesContractTests
{
// cfb not available on windows 7
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.Encryption.Des.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static partial class DesTests
{
private static readonly byte[] KnownWeakKey = "e0e0e0e0f1f1f1f1".HexToByteArray();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class DSAImportExport
{
public static bool SupportsFips186_3 => DSAFactory.SupportsFips186_3;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DSAKeyFileTests
{
public static bool SupportsFips186_3 => DSAFactory.SupportsFips186_3;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class DSAKeyGeneration
{
public static bool SupportsKeyGeneration => DSAFactory.SupportsKeyGeneration;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DSAKeyPemTests
{
private const string AmbiguousExceptionMarker = "multiple keys";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class DSASignVerify_Array : DSASignVerify
{
public override byte[] SignData(DSA dsa, byte[] data, HashAlgorithmName hashAlgorithm) =>
Expand Down Expand Up @@ -54,7 +54,7 @@ public void InvalidStreamArrayArguments_Throws()
}
}

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class DSASignVerify_Stream : DSASignVerify
{
public override byte[] SignData(DSA dsa, byte[] data, HashAlgorithmName hashAlgorithm) =>
Expand All @@ -76,7 +76,7 @@ public void InvalidArrayArguments_Throws()
}

#if NETCOREAPP
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class DSASignVerify_Span : DSASignVerify
{
public override byte[] SignData(DSA dsa, byte[] data, HashAlgorithmName hashAlgorithm) =>
Expand Down Expand Up @@ -109,7 +109,7 @@ private static byte[] TryWithOutputArray(Func<byte[], (bool, int)> func)
}
}
#endif
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract partial class DSASignVerify
{
public abstract byte[] SignData(DSA dsa, byte[] data, HashAlgorithmName hashAlgorithm);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract class DSASignatureFormatTests : DsaFamilySignatureFormatTests
{
protected override bool SupportsSha2 => DSAFactory.SupportsFips186_3;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class DSASignatureFormatterTests : AsymmetricSignatureFormatterTests
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.Dsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class DSAXml
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

namespace System.Security.Cryptography.Algorithms.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract class DsaFamilySignatureFormatTests
{
protected readonly struct KeyDescription
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

namespace System.Security.Cryptography.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract partial class ECKeyFileTests<T> where T : AsymmetricAlgorithm
{
protected abstract T CreateKey();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract class ECKeyPemTests<TAlg> where TAlg : AsymmetricAlgorithm
{
private const string AmbiguousExceptionMarker = "multiple keys";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.EcDiffieHellman.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public class ECDhKeyFileTests : ECKeyFileTests<ECDiffieHellman>
{
protected override ECDiffieHellman CreateKey()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class ECDiffieHellmanKeyPemTests : ECKeyPemTests<ECDiffieHellman>
{
protected override ECDiffieHellman CreateKey() => ECDiffieHellman.Create();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ public static void TestKeySizeCreateKey()
}

[Fact]
[PlatformSpecific(~TestPlatforms.Android)] // Android does not validate curve parameters
[SkipOnPlatform(TestPlatforms.Android, "Android does not validate curve parameters")]
public static void TestExplicitImportValidationNegative()
{
if (!ECDiffieHellmanFactory.ExplicitCurvesSupported)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

namespace System.Security.Cryptography.EcDiffieHellman.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class ECDiffieHellmanTests : EccTestBase
{
private static List<object[]> s_everyKeysize;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public class ECDsaImportExportTests : ECDsaTestsBase
{
internal static bool CanDeriveNewPublicKey { get; }
Expand Down Expand Up @@ -188,7 +188,7 @@ public static void TestKeySizeCreateKey()
}

[ConditionalFact(nameof(ECExplicitCurvesSupported))]
[PlatformSpecific(~TestPlatforms.Android)] // Android does not validate curve parameters
[SkipOnPlatform(TestPlatforms.Android, "Android does not validate curve parameters")]
public static void TestExplicitImportValidationNegative()
{
unchecked
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public class ECDsaKeyFileTests : ECKeyFileTests<ECDsa>
{
protected override ECDsa CreateKey()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class ECDsaKeyPemTests : ECKeyPemTests<ECDsa>
{
protected override ECDsa CreateKey() => ECDsa.Create();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract class ECDsaSignatureFormatTests : DsaFamilySignatureFormatTests
{
protected override bool SupportsSha2 => true;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class ECDsaTests_Array : ECDsaTests
{
protected override bool VerifyData(ECDsa ecdsa, byte[] data, int offset, int count, byte[] signature, HashAlgorithmName hashAlgorithm) =>
Expand Down Expand Up @@ -87,7 +87,7 @@ public void VerifyHash_InvalidArguments_Throws(ECDsa ecdsa)
}
}

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class ECDsaTests_Stream : ECDsaTests
{
protected override bool VerifyData(ECDsa ecdsa, byte[] data, int offset, int count, byte[] signature, HashAlgorithmName hashAlgorithm)
Expand Down Expand Up @@ -125,7 +125,7 @@ public void VerifyData_InvalidArguments_Throws(ECDsa ecdsa)
}
}

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract partial class ECDsaTests : ECDsaTestsBase
{
protected bool VerifyData(ECDsa ecdsa, byte[] data, byte[] signature, HashAlgorithmName hashAlgorithm) =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public sealed class ECDsaTests_Span : ECDsaTests
{
protected override bool VerifyData(ECDsa ecdsa, byte[] data, int offset, int count, byte[] signature, HashAlgorithmName hashAlgorithm) =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ namespace System.Security.Cryptography.EcDsa.Tests
/// <summary>
/// Input and helper methods for ECDsa
/// </summary>
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public abstract class ECDsaTestsBase : EccTestBase
{
#if NETCOREAPP
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

namespace System.Security.Cryptography.EcDsa.Tests
{
[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public partial class ECDsaXml : ECDsaTestsBase
{
[Fact]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ namespace System.Security.Cryptography.Encryption.RC2.Tests
{
using RC2 = System.Security.Cryptography.RC2;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
[ConditionalClass(typeof(RC2Factory), nameof(RC2Factory.IsSupported))]
public static class RC2CipherTests
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ namespace System.Security.Cryptography.Encryption.RC2.Tests
{
using RC2 = System.Security.Cryptography.RC2;

[SkipOnMono("Not supported on Browser", TestPlatforms.Browser)]
[SkipOnPlatform(TestPlatforms.Browser, "Not supported on Browser")]
public static class RC2ContractTests
{
[Theory]
Expand Down
Loading

0 comments on commit bc9f00c

Please sign in to comment.