Merge upstream runtime main to feature/s390x #884
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Refer activity guide to the official docs dotnet/docs#23313 is bringing more enhancements shortly
- Unify workhorse implementations across all inbox encoders - Refactor most unsafe code from TextEncoder workhorse routines into standalone helpers - Fix bounds check logic in workhorse routines - SSSE3-optimize central workhorse routine - Remove vestigial code from the library and unit test project - Add significant unit test coverage for the workhorse routines and unsafe helpers - Ref: CVE-2021-26701 (MSRC 62749)
* Fix or disable warnings on latest VS dogfood * Delete JIT local warning disables
* Fix warning for System.IO.Compression.Native when more analysis is enabled. * Fix warnings in corehost when more analysis is enabled.
- Implementation is parallel to existing ibc handling, so that it can be toggled on/off by adjusting the `UsingToolIbcOptimization` property - Use the same data for all assemblies produced in current build - Apply data to release builds only - Disable mismatch assertions in jit for current state where il mismatches are common
Before this fix, when doing replay (not asmdiffs) of a set of MCH files, the `-f` arguments would accumulate, one for every MCH file run so far.
* Use TryGetBits to avoid dependency on accessing internal fields of decimal type (which would be endian-dependent)
* Always use little-endian encoding to compute checksum
* Skip Xoshiro_AlgorithmBehavesAsExpected on big-endian systems Co-authored-by: Stephen Toub <stoub@microsoft.com>
* Fix Utf8Formatter.Guid on big-endian systems (first three fields are stored in native byte order, not always little-endian) * Use Guid constructor in BlobContentId.FromHash to remove endian-dependent code accessing the structure directly
* Add new subset to build only the IL tools (ilasm/ildasm)
(enabled by default)
* New -skipiltools argument to src/coreclr/build-runtime.{cmd,sh}
(skips building the IL tools if present)
* Actually make -skipjit argument work to not build the JIT & VM
* Take byte order into accout when packing characters into an int
…9812) * BlobBuilder: do not use Encoding.Unicode.GetBytes but simple byte-swaps on big-endian systems (fixes a semantic difference to little-endian code w.r.t. surrogate char handling) * BlobWriter: add big-endian WriteUTF16 code path like BlobBuilder
* Byte-swap characters read from memory-mapped resources on big-endian systems
Microsoft.DotNet.XHarness.CLI , Microsoft.DotNet.XHarness.TestRunners.Xunit From Version 1.0.0-prerelease.21165.2 -> To Version 1.0.0-prerelease.21169.1 Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>
* Slimmer IOptions<T> - Use a Lazy instead of the IOptionsCache (which is a concurrent dictionary)
* Add additional tests. * Use LclFld when copy to/from promoted from/to an unpromoted struct/block. * Response review.
…l architectures (#49734) - Use arm jit for armel in crossgen2, and build the matching bits on all architectures
- Crossgen2 computes its signatures from slightly different data than the VM does - Tweak the jit interface to specify that the resolved token comes from devirtualization to detect this special case - The extra tokenType information is ignored by the VM - Pass the devirtualization specified owning type through when necessary - Check that it meets the requirements of being defined on a a derived type from the method target
…aint satisfaction (#47258) - The constraint processing logic in the runtime conflates the idea of constraint checking on open, closed over concrete types, and closed over non-concrete types - This change adds a tweak to avoid using an instantiation context that isn't related to the type variable being instantiated Fixes issue #45600 and adds a regression test from the customer.
Previously, we have had four iOS RIDs:
iOS-arm
iOS-arm64
iOS-x86
iOS-x64
Apple has never shipped an iOS device with an x86 or x64 processor. Instead, the x86/x64 RIDs have meant "iOS simulator with these arches" as opposed to "iOS with these arches". Amongst other things, that means building against a DIFFERENT SDK, iPhoneSimulator.platform vs iPhoneOS.platform
In the Apple Silicon future, the iOS simulator is now an ARM64 binary - so we need:
iOS-arm
iOS-arm64
iOS-arm64, but built against the simulator SDK not the device SDK
iOS-x86
iOS-x64
Clearly, there's a problem.
The solution is to move the simulators to a different RID, to avoid the overloading issue:
iOS-arm
iOS-arm64
iOSSimulator-arm64
iOSSimulator-x86
iOSSimulator-x64
This PR introduces the new entries in the RID graph, moves our existing iOS-x{86,64} to iOS-sim-x{86,64}, adds a new iOS-arm64.
The above also applies for tvOS, with a smaller set of OSes:
tvOS-arm64
tvOSSimulator-arm64
tvOSSimulator-x64
Ref: #48216
* Resolving ILLink warnings on System.Private.Xml (Part 1) * Only annotate the unsafe Load overload methods from XslCompiledTransform type * Address feedback and fix one more warning * Update src/libraries/System.Private.Xml/src/System/Xml/Xsl/Runtime/EarlyBoundInfo.cs Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com> * PR Feedback Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
* Remove unnecessary suppressions of IL2060 These suppressions are no longer necessary with the latest trimmer version. Fix #49486
…cate stores (#48862)
``` cd emsdk && ./emsdk install `cat emscripten-version.txt` cat: emscripten-version.txt: No such file or directory ```
(*) When compiling tests with Crossgen2, use optimizations by default like Crossgen1 does. (*) Don't use large version bubble when Crossgen2-compiling the framework. While at some point this had value as preparatory testing for composite mode, it's not a shipping scenario right now and it was hiding the codegen bug causing the issue dotnet/runtime#49826 in System.Text.Json. Thanks Tomas
* Convert all configuration options from REGUTIL to CLRConfig. * Remove uses of REGUTIL outside of CLRConfig impl.
* versioning * no need to support 2.0 bundle version * enable compression * fixes after rebase * build fix for Unix * Fix build with GCC * disable a test temporarily to see what else breaks * Add EnableCompression option to Bundler + PR feedback * Couple more places should use version 6.0 * PR feedback (header versioning, more tests, fixed an assert) * Suggestion from PR review Co-authored-by: Vitek Karas <vitek.karas@microsoft.com> * sorted usings * should be bundle_major_version in two more places. * More PR feedback Co-authored-by: Vitek Karas <vitek.karas@microsoft.com>
Adjust the error code determination for asm diffs to not report diffs if all the compilation failures were due to missing data in the MCH files. Update the SuperPMI end status line to report the count of MC compilation failures due to missing data (and adjust parallel SuperPMI to handle it as well). As a result, with `superpmi.py asmdiffs` you won't see the message "Asm diffs" if all the failures were due to missing data.
- Split browser from non-browser to avoid CA1416 suppressions, UnsupportedOSPlatform attributes, and PNSEs when used from browser - Cache domain name and local addresses for use in IsLocal, along with clearing of the caches upon network change detection - Removing redundant IPAddress.IsLoopback check that's covered by previous Uri.IsLoopback check - Change IsMatchInBypassList to use string interpolation, such that the non-default port case will get better implicitly with C# 10 interpolated string improvements
* Add CancellationTokenSource.TryReset * Update src/libraries/System.Private.CoreLib/src/System/Threading/CancellationTokenSource.cs
* Do not morph away double negation if it is a CSE candidate * Added a test verifying that double negation is not removed for CSEs * Bump the test's priority to zero
…essConstructor (#50390) This allows private constructors on the Types to be trimmed. Fix #50353
* ignore empty domain for digets auth * style cleanup
* Trim TimeZoneInfo to reduce wasm size * Use partial class files instead of ifdefs
Co-authored-by: Matt Galbraith <MattGal@users.noreply.github.com>
* mono_arch_unwind_frame: Handle FPRs during unwind. (This is important since GCC will spill GPRs to FPRs instead of the stack when beneficial.) * map_hw_reg_to_dwarf_reg: Implement DWARF register number mapping for FPRs. (https://github.com/IBM/s390x-abi/releases/download/v1.5/lzsabi_s390x.pdf) * mono_arch_create_generic_trampoline: Fix incorrect start address.
…to-s390x # Conflicts: # eng/Subsets.props # eng/Versions.props # src/libraries/System.Private.CoreLib/src/System.Private.CoreLib.Shared.projitems
### Small s390x fixes - Add another tailcall disqualifying condition - Minor formatting change ### New Array Parameter fix We have been seeing msbuild tasks fail due to an OverflowException being raised in `(wrapper_alloc)_object:AllocVector` because the number of elements appears to be negative (0xffffffff00000088). However, this is just the result of a missing type conversion somehow. The original C# code is simply: ``` _metadataStringTable = new string[_reader.ReadInt32()]; ``` The IL corresponding to that line reads: ``` converting (in B2: stack: 2) IL_0021: callvirt 0x0a00035e cmethod = int System.IO.BinaryReader:ReadInt32 () Call requires: 1 parameters converting (in B2: stack: 2) IL_0026: newarr 0x010000b6 ``` Note that the return from the `ReadInt32` call is of type `int` (i.e. 32 bits) and is passed unmodified as argument to `newarr`. This remains the same in the initial stages of Mono JIT compilation: ``` call_membase R43 <- [R45 + 0xe8] [int System.IO.BinaryReader:ReadInt32 ()] [s390_r2 <- R44] clobbers: c il_seq_point il: 0x26, nonempty-stack newarr R46 <- R43 ``` But later the `newarr` is implemented in terms of a function call: ``` call_membase R43 <- [R45 + 0xe8] [int System.IO.BinaryReader:ReadInt32 ()] [s390_r2 <- R44] clobbers: c il_seq_point il: 0x26, nonempty-stack i8const R81 <- [2929315620864] move R83 <- R81 move R84 <- R43 call R46 <- [(wrapper alloc) object object:AllocVector (intptr,intptr)] [s390_r2 <- R83] [s390_r3 <- R84] clobbers: c ``` And here the argument is of type `intptr` (i.e. 64 bits). But the return value of ReadInt32 is still passed through directly to `AllocVector` without any conversion, and this remains true in the final assembler code: ``` b2: 0d e1 basr %r14,%r1 // ReadInt32 b4: b9 04 00 32 lgr %r3,%r2 // <<- simple move of the return value b8: c0 28 00 00 02 aa iihf %r2,682 be: c0 29 08 d1 28 00 iilf %r2,147924992 c4: c0 e8 00 00 03 ff iihf %r14,1023 ca: c0 e9 9d ec 97 08 iilf %r14,2649528072 d0: 0d ee basr %r14,%r14 // AllocVector ``` This becomes a problem because the implementation of `AllocVector` does indeed expect a 64-bit input, and throws an exception if that is negative. In addition, the implementation of `ReadInt32` only computes a 32-bit result in the low 32 bits of the register, and leaves the upper half undefined. Due to the particular code sequence we get a return with the upper half nonzero even though the lower half is a positive integer: ``` e6: e3 20 f0 e0 00 14 lgf %r2,224(%r15) <<- if this is a 32-bit value with the top bit set, then the upper half of %r2 is now 0xffffffff ec: e3 20 f0 b8 00 50 sty %r2,184(%r15) f2: b9 04 00 32 lgr %r3,%r2 <<- and so is %r3 f6: c0 01 00 ff 00 ff lgfi %r0,16711935 fc: b9 e4 00 43 ngrk %r4,%r3,%r0 100: b9 14 00 24 lgfr %r2,%r4 104: 88 20 00 08 srl %r2,8 108: 89 40 00 18 sll %r4,24 10c: b9 e6 40 22 ogrk %r2,%r2,%r4 110: c0 01 ff 00 ff 00 lgfi %r0,-16711936 <<-- the upper half of %r0 is also 0xffffffff 116: b9 e4 00 43 ngrk %r4,%r3,%r0 <<-- and so is %r4 11a: b9 14 00 34 lgfr %r3,%r4 11e: 89 30 00 08 sll %r3,8 122: 88 40 00 18 srl %r4,24 126: b9 e6 40 33 ogrk %r3,%r3,%r4 <<- and now %r3 12a: b9 08 00 23 agr %r2,%r3 <<- and therefore %r2, where the upper half for 0 before 12e: e3 20 f0 b8 00 50 sty %r2,184(%r15) 134: a7 fb 00 f8 aghi %r15,248 138: eb 6e f0 30 00 04 lmg %r6,%r14,48(%r15) 13e: 07 fe br %r14 <<- and here it is returned unchanged ```
|
Thanks for doing the merge @akoeplinger ! With this version, it should be possible to remove all test exclusions except for the crypto ones (unless #843 is also merged). |
|
Interestingly we have a few System.Drawing.Common tests fail with this merge. Maybe something changed with the libgdiplus version on the bots? |
|
Will merge this and disable System.Drawing.Common in #679 and reenable the rest. |
|
Hmm, I've seen the System.Drawing.Common failures when Mono is also installed on the machine; they seem to go away when it is uninstalled. Not sure if this happened on your build machines ... In any case, I agree we should disable the test for now; I'll have a look and see if I can fix it. |
|
Ah ok, that makes sense. The System.Drawing.Common tests check whether libgdiplus is available and skips tests otherwise. I assume somebody installed mono (which includes libgdiplus) on the bots. |
|
Huh, interesting. Does this mean dotnet relies on (parts of) mono being available for some of its functionality, then? |
|
Hi @akoeplinger I've now fixed all failures in the System.Drawing.Common tests. However, the only bug in dotnet code is this one: The majority of the failures are actually caused by endian bugs in libgdiplus itself. I've submitted pull requests here: In addition, while not necessary to make the dotnet tests pass, when running the libgdiplus test suite I also found these: |
|
Unfortunately this means that even when the libgdiplus fixes are merged upstream, we still need to keep the dotnet test disabled until the fixes are available in Linux distros (at least in the distro on the test machine). |
|
@uweigand thanks, I've now merged the PRs :) |
|
Thanks @akoeplinger ! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
/cc @uweigand @nealef