Added 'standalone' option when building tests for ilasm roundtrip

[TIHan]

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Current pipeline run: https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results

[ghost]

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Set the environment variable BuildAsStandalone=true when we are building tests for ilasm roundtripping.

Author:	TIHan
Assignees:	TIHan
Labels:	area-Infrastructure-coreclr
Milestone:	-

[markples]

Can you please show a lab run with this ilasm flag running?

[markples]

I think you should create an $(ilasm) so that you don't duplicate the entire script line. It will be easy for this to go stale.

[markples]

I have some concerns about doing this. I think you may run into trouble with the "two pass" nature of building tests, but testing ilasm will determine if that is an actual issue. But also I'm not sure that you can just change the build arguments and still have coherent pipelines. For example, the artifact name is going to be the same. This means that normal and ilasm builds can't be in the same pipeline, which perhaps is already the case but would be an unfortunate dependency. Also, typically one has expectations about what is in a particular artifact based on the name. There may be additional issues with the overall pipelines so it would be good to get a review from someone more knowledgeable than me.

[TIHan]

Can you please show a lab run with this ilasm flag running?

https://dev.azure.com/dnceng-public/public/_build/results?buildId=427945&view=results should be that

[markples]

I don't see any changes compared to normal. The test artifacts zip has main-less dlls, and the test runs appear to be ildasm/ilasm-ing the test executors and then calling tests in dlls.

[TIHan]

@markples This is what I get when I compile a test as standalone and do run.cmd x64 checked ilasmroundtrip. I guess the test wrapper is still referring to the old assembly?

[markples]

I don't know what you mean about the test wrapper. I see no evidence that your change has changed any behavior in the lab.

[TIHan]

I don't know what you mean about the test wrapper

Meaning the test wrapper is dependent on Runtime_92590.dll and not Runtime_92590.asm.dll.

[markples]

.asm.dll is created during test execution in the .cmd/.sh files. The new wrappers depend on the .dlls for intra-proc tests, but the build system has no knowledge of the .asm.dll thing happening at test execution time. The new wrappers and the old infrastructutre depend on calling the .cmd/.sh files and have no knowledge of what is being executed within them.

However, the problem here is deeper than that. I don't see entry points in the individual tests outside of ones that normally get them via RequiresProcessIsolation. There isn't anything for the roundtripping infrastructure to invoke even if it tried.

[TIHan]

@markples since I merged #90110 - this will actually replace the original assembly in its original location, so that might mean this could work? i.e. there are no .asm.dlls being created alongside the original .dll.

[markples]

I don't see how that matters. I still don't think the roundtripping logic is even being called. If you look at any of the logs, you'll see something like this:

01:14:15.219 Running test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
PASSED
01:14:15.222 Passed test: JIT/Directed/coverage/oldtests/zeroinit_r/zeroinit_r.dll
01:14:15.227 Running test: JIT/Directed/coverage/flowgraph/gcpoll/gcpoll.cmd

Return code:      0
Raw output file:      C:\h\w\AABC097D\w\ACF4097A\uploads\coverage\flowgraph\gcpoll\output.txt
Raw output:
BEGIN EXECUTION
C:\h\w\AABC097D\p\ildasm.exe /raweh /unicode /out=gcpoll.dasm.il gcpoll.dll

For gcpoll, you see the ildasm logic kick in. This is due to it being a RequiresProcessIsolation test. But the tests before it, like zeroinit_r aren't doing that.

[TIHan]

Well, it's now getting errors with EXEC : error : No entry point declared for executable [C:\work\runtime\src\tests\Regressions\coreclr\22021\provider.ilproj] [C:\work\runtime\src\tests\build.proj]

[TIHan]

Ok, I see that is only does it for RequiresProcessIsolation. It's interesting because if I only build a single test that doesn't have RequiresProcessIsolation, and I have BuildAsStandalone=true, I can run the roundtrip test on it.

[markples]

2 things going on here

• Individual tests work locally but not in the lab because this PR is not sufficient to set up the lab.
• I believe that the break was exposed by test merging of src\tests\Regression, which removed the OutputType=Library of that test, which was following in the footsteps of my CLRTestKind cleanup. The global setting BuildAsStandalone can't tell the difference between a normal test and a library for a test. (A normal test is a library except that RequiresProcessIsolation and BuildAsStandalone add an entry point and promote it to an executable.)

[TIHan]

@markples I think this is now running the roundtrip tests, though failing in Unix, the Windows versions passed: https://dev.azure.com/dnceng-public/public/_build/results?buildId=432469&view=logs&j=bc936583-455b-59a6-ba6a-b6595d929238&t=b93c0282-5af1-5880-fe4a-b9ec04a10b26

[TIHan]

Ah, but now I see it only did it for RequiresProcessIsolation tests. Just checked Kusto and looked at the JitBlue tests and there are only a handful of them. Which was said before, this happens by default; means the standalone didn't do the right thing.

[TIHan]

Closing in favor of #93368