Support O_DIRECT file open option handling for FileStream.Unix

[buybackoff]

As discussed in #19229, changes are very small. The value 0x4000 is from Mono and cross-checked with some Google search.

[dnfclas]

All CLA requirements met.

[jkotas]

Do we need to have matching change in System.Native and tests to verify that this works (ie does not crash)?

Also, since this is de-facto public surface now, should we expose it as enum value in reference assembly so that folks do not need to use a magic 0x20000000 constant?

[buybackoff]

The idea was that this is not public and not officially supported, but did work for Windows and nice to have on Linux to avoid OS page cache trashing. I do not think it is public or officially supported now, just extending Windows hidden hack to Linux. E.g. to support it on MacOS requires some work with fcntl, but on Linux it's a single bit to match the existing Windows behavior. The modified file is already specific to Unix/Linux, I do not propose to add proper nobuffering support on all platforms.

…

On Thu, Aug 2, 2018, 1:47 AM Jan Kotas ***@***.***> wrote: Do we need to matching change in System.Native and tests to verify that this works (ie does not crash)? Also, since this is de-facto public surface now, should be expose it as enum value in reference assembly? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19232 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAd5fLDIKbRuZPE8mzT98P_Vk3uU5cpUks5uMi_kgaJpZM4VrRqh> .

[buybackoff]

The PR was by far the fastest way to test accross different distros, the flag was in the tests already. The test is failing only on MacOs and one of Ubuntus, so some work with System.Native is needed. I would be happy if it only worked on CentOS (AWS) and ignored on others, but now it doesn't look so simple. Since FileStream accepts raw file handle a single PInvoke to open a file with proper options is probably simpler.

[jkotas]

so some work with System.Native is needed

Right, you need to add it to these places first - the value of the flag should be the next available one (0x0100):

https://github.com/dotnet/corefx/blob/master/src/Native/Unix/System.Native/pal_io.h#L141
https://github.com/dotnet/corefx/blob/master/src/Native/Unix/System.Native/pal_io.c#L254
https://github.com/dotnet/corefx/blob/master/src/Native/Unix/System.Native/pal_io.c#L270

failing only on MacOs and one of Ubuntu

That's because of these are the only Unix flavors that run the relevant tests in CoreCLR CI. The test would fail on any Unix.

[buybackoff]

@jkotas
Thanks for the direction, I would be lost otherwise. Will update this PR later and do local tests before pushing.

[GSPP]

Maybe the 0x20000000 value should be supported and documented now. It is effectively supported anyway because breaking it would break to much real world code. It seems like a useful thing to have on all platforms. Database-like applications often have a hard requirement on having this.

[buybackoff]

I have built CoreClr/CoreFx with O_DIRECT changes on WSL 18.04 and was able to run my test program using the build explicitly (~/dev/dotnet/coreclr/Tools/dotnetcli/dotnet run -c Release), so it at least compiles and runs. But it looks like even WriteThrough is ignored, only Flush/fsync works as expected. There is no disk activity and my benchmark numbers are almost the same as on Windows with FileOptions.None.

I have several questions:

1. Should I submit a PR with native changes to corefx repo and close this one?
2. How to run corefx test without rebuilding everything? I try build-test.sh -release, it takes a long time to build and fails with messages like error CS0006: Metadata file '/mnt/c/MD/Dev/dotnet/corefx/bin/ref/netcoreapp/System.IO.Pipelines.dll' could not be found or error MSB3030: Could not copy the file "/mnt/c/MD/Dev/dotnet/corefx/bin/AnyOS.AnyCPU.Release/PerfRunner/netstandard/PerfRunner.pdb" because it was not found.
3. Do you know if VirtualBox with Ubuntu supports O_SYNC/O_DIRECT or I need native Linux install?

BTW, On Windows performance of the same code differs by near 30% with 1.2 GB/sec with WriteThrough+NoBuffering vs 900MB/sec WriteThrough-only on a consumer-grade NVMe SSD (EVO 960 250G, max sequential write 1.46GB/sec), which I would say is massive.

[jkotas]

Should I submit a PR with native changes to corefx repo

Yes.

Also, could you please create API proposal in CoreFX repo for adding the flag to the FileOptions enum?

close this one?

I think so.

How to run corefx test without rebuilding everything?

I do not think there is a way to do that. https://github.com/dotnet/corefx/blob/master/Documentation/project-docs/developer-guide.md#building-individual-libraries says that you have to build the whole thing once.

Do you know if VirtualBox with Ubuntu supports O_SYNC/O_DIRECT or I need native Linux install?

My guess is that it will support it, but I am not sure whether you can trust the performance measurements made on VirtualBox.