Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix module init issue with Verify #2092

Merged
merged 3 commits into from
Dec 19, 2022
Merged

Fix module init issue with Verify #2092

merged 3 commits into from
Dec 19, 2022

Conversation

mattjohnsonpint
Copy link
Contributor

We've started getting intermittent issues such as the following. Run the same test repeatedly and it fails about half the time and passes about half the time.

Test run for /Users/matt/Code/getsentry/sentry-dotnet/test/Sentry.AspNetCore.Grpc.Tests/bin/Release/net6.0/Sentry.AspNetCore.Grpc.Tests.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.4.0 (arm64)
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
[xUnit.net 00:00:00.73]     Sentry.AspNetCore.Grpc.Tests.IntegrationTests.UnhandledException_AvailableThroughLastExceptionFilter [FAIL]
  Failed Sentry.AspNetCore.Grpc.Tests.IntegrationTests.UnhandledException_AvailableThroughLastExceptionFilter [1 ms]
  Error Message:
   System.TypeInitializationException : The type initializer for '<Module>' threw an exception.
---- System.Exception : The API 'VerifyTests.VerifyHttp.Enable' must be called prior to any Verify has run. Usually this is done in a [ModuleInitializer].
  Stack Trace:
     at System.RuntimeTypeHandle.GetActivationInfo(ObjectHandleOnStack pRuntimeType, (fnptr)* ppfnAllocator, Void** pvAllocatorFirstArg, (fnptr)* ppfnCtor, BOOL* pfCtorIsPublic)
   at System.RuntimeTypeHandle.GetActivationInfo(RuntimeType rt, (fnptr)& pfnAllocator, Void*& vAllocatorFirstArg, (fnptr)& pfnCtor, Boolean& ctorIsPublic)
   at System.RuntimeType.ActivatorCache..ctor(RuntimeType rt)
   at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean wrapExceptions)
   at System.Activator.CreateInstance(Type type, Boolean nonPublic, Boolean wrapExceptions)
   at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
   at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes)
   at System.Activator.CreateInstance(Type type, Object[] args)
   at ReflectionAbstractionExtensions.<>c__DisplayClass0_0.<CreateTestClass>b__0() in /_/src/xunit.execution/Extensions/ReflectionAbstractionExtensions.cs:line 42
   at ReflectionAbstractionExtensions.CreateTestClass(ITest test, Type testClassType, Object[] constructorArguments, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) in /_/src/xunit.execution/Extensions/ReflectionAbstractionExtensions.cs:line 42
----- Inner Stack Trace -----
   at VerifyTests.InnerVerifier.ThrowIfVerifyHasBeenRun() in /_/src/Verify/Verifier/InnerVerifier.cs:line 27
   at VerifyTests.VerifierSettings.RegisterFileConverter[T](Conversion`1 conversion, CanConvert`1 canConvert)
   at VerifyTests.VerifyHttp.Enable() in C:\projects\verify-http\src\Verify.Http\VerifyHttp.cs:line 25
   at ModuleInit.Init() in /Users/matt/Code/getsentry/sentry-dotnet/test/Sentry.AspNetCore.Tests/ModuleInit.cs:line 6
   at .cctor()

Failed!  - Failed:     1, Passed:     5, Skipped:     0, Total:     6, Duration: 78 ms - Sentry.AspNetCore.Grpc.Tests.dll (net6.0)

After some investigation, I believe this is due to a recently added Verify feature: VerifyTests/Verify#745.

It would seem that this fails in the case of test project referencing another test project. In our case, the Sentry.AspNetCore.Tests project is referenced by three others. Since the order tests run is nondeterministic, if a test that uses the parent project runs first, then the parent project's module init is called before any verify tests and everything is fine. But if one of the child project's verify tests that doesn't use the parent project is run first (such as the ApiApprovalTests), then the module init of the parent project runs too late and throws the error.

@SimonCropp - this should probably be addressed in Verify itself.

As a workaround, we can just make sure each child project has its own module init that uses something from the parent project - which gets the parent project's module init to run immediately.

#skip-changelog

@mattjohnsonpint
Copy link
Contributor Author

Looks like the same thing is happening for VerifyDiffPlex.Initialize, but only on Windows for some reason...

  Failed Sentry.Tests.Internals.MemoryInfoTests.WriteTo [1 ms]
  Error Message:
   System.TypeInitializationException : The type initializer for '<Module>' threw an exception.
---- System.Exception : The API 'VerifyTests.VerifyDiffPlex.Initialize' must be called prior to any Verify has run. Usually this is done in a [ModuleInitializer].
  Stack Trace:
     at Sentry.Tests.Internals.MemoryInfoTests..ctor(ITestOutputHelper output)
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.ConstructorInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
----- Inner Stack Trace -----
   at VerifyTests.InnerVerifier.ThrowIfVerifyHasBeenRun() in /_/src/Verify/Verifier/InnerVerifier.cs:line 27
   at VerifyTests.VerifyDiffPlex.Initialize(OutputType outputType) in C:\projects\verify-diffplex\src\Verify.DiffPlex\VerifyDiffPlex.cs:line 13
   at VerifyTests.VerifyDiffPlex.Initialize() in C:\projects\verify-diffplex\src\Verify.DiffPlex\VerifyDiffPlex.cs:line 9
   at ModuleInit.Init() in D:\a\sentry-dotnet\sentry-dotnet\test\Sentry.Testing\ModuleInit.cs:line 6
   at .cctor()

@mattjohnsonpint
Copy link
Contributor Author

mattjohnsonpint commented Dec 19, 2022

I was able to do a small repro project that fails every time: https://github.com/mattjohnsonpint/repro-verify-http-issue

But interestingly, it also gives CA2255 warning on the initializer in the base class. Not sure why we don't get one in Sentry.Testing.ModuleInit, but we should.

CA2255: The ModuleInitializer attribute should not be used in libraries

Thus the real problem here isn't Verify, but that we're initializing parts of verify in the base libraries rather in that the end test project. I'll make an update to resolve that.

@mattjohnsonpint
Copy link
Contributor Author

New approach:

  • Created Sentry.Tests.AspNetCore.TestUtils.

    • Moved anything shared out of SentryTests.AspNetCoreTests over, and updated all references and namespaces.
  • Created a CommonModuleInit.cs and linked it to all test projects.

    • Move init code from Sentry.Testing.ModuleInit to this file.

Net result should be only the final test projects use module initializers - referenced projects won't have any.

@mattjohnsonpint mattjohnsonpint changed the title Fix module init issue with VerifyHttp Fix module init issue with Verify Dec 19, 2022
@mattjohnsonpint mattjohnsonpint merged commit 9840ccb into main Dec 19, 2022
@mattjohnsonpint mattjohnsonpint deleted the fix-moduleinit branch December 19, 2022 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants