implement O_DIRECT (NO_BUFFERING) support for Unix #55463
Conversation
|
Tagging subscribers to this area: @dotnet/area-system-io Issue DetailsWith the introduction of the new Since we already support it on Windows by simply allowing the users to pass magic We could do it on Unix as well.
|
Should we add this value to the public API surface? |
| @@ -263,6 +263,8 @@ static int32_t ConvertOpenFlags(int32_t flags) | |||
| ret |= O_TRUNC; | |||
| if (flags & PAL_O_SYNC) | |||
| ret |= O_SYNC; | |||
| if (flags & PAL_O_DIRECT) | |||
| ret |= O_DIRECT; | |||
There was a problem hiding this comment.
The man page states:
O_DIRECT I/Os should never be run concurrently with the fork(2)
system call, if the memory buffer is a private mapping (i.e., any
mapping created with the mmap(2) MAP_PRIVATE flag; this includes
memory allocated on the heap and statically allocated buffers).
Any such I/Os, whether submitted via an asynchronous I/O
interface or from another thread in the process, should be
completed before fork(2) is called. Failure to do so can result
in data corruption and undefined behavior in parent and child
processes.
How are you guaranteeing this, in our tests but also for use in the wild? fork is used by Process.Start.
| public unsafe void NoBufferingOptions(FileOptions option) | ||
| { | ||
| void* pWrite = NativeMemory.AlignedAlloc(c_DefaultBufferSize, c_DefaultBufferSize); | ||
| void* pRead = NativeMemory.AlignedAlloc(c_DefaultBufferSize, c_DefaultBufferSize); |
There was a problem hiding this comment.
Why two buffers rather than one?
There was a problem hiding this comment.
According to the man page:
the alignment of the user
buffer, and the file offset must all be multiples of the logical
block size of the filesystem
How are you guaranteeing that this 4096 alignment is valid everywhere this test runs?
I'm also not clear how this works with the loop that is here. Is the implementation guaranteed to successfully read/write in multiples of the alignment such that the loop will use a correctly aligned offset at every iteration?
|
I am going to close the PR right now and re-start the discussion when I am back from vacations. |
ghost
locked as resolved and limited conversation to collaborators
With the introduction of the new
NativeMemoryAPI it's much easier to perform direct file IO with .NET.Since we already support it on Windows by simply allowing the users to pass magic
(FileOptions)0x20000000asFileOptions:runtime/src/libraries/System.Private.CoreLib/src/System/IO/FileStream.cs
Line 166 in 2223bab
We could do it on Unix as well.