Open keywords

[bramtayl]

No description provided.

[bramtayl]

Part 3 of #25188, cf #3558

[StefanKarpinski]

You might want to wrap the line here.

[StefanKarpinski]

This overwrites the above method with a different set of keyword arguments.

[StefanKarpinski]

There should be some relationship between truncate and append. Probably truncate = !append. Otherwise the behavior of open(file, append=true) is going to be rather surprising. I also suspect that write = append makes sense. Similarly, create typically default to true if write is set. I think you'll need to draw a graph of the relationships here to figure this one out – this is why this change hasn't been done yet.

[StefanKarpinski]

I went ahead and did it, although this probably could have been done without the image:

So it looks like the signature should be something like this:

open(fname::AbstractString;
    append::Bool = false,
    write::Bool = append,
    read::Bool = !write,
    create::Bool = write,
    truncate::Bool = !append,
)

[StefanKarpinski]

Actually, that's not quite right since you want truncate = write && !append.

[bramtayl]

ok so

open(fname::AbstractString;
    append::Bool = false,
    write::Bool = append,
    read::Bool = !write,
    create::Bool = write,
    truncate::Bool = write && !append,
)

?

[StefanKarpinski]

Actually, I think we may well have a bunch of these implications a bit backwards. I'm going to see if I can come up with a more sensible way to define the defaults based on what they mean.

[StefanKarpinski]

This is way trickier than I imagined at first if you want it to have intuitive "do what I mean" behavior. I took the table of fopen modes, together how we might want to concisely express them using boolean keywords and the actual flags that they use as a set of motivating examples:

Mode	Keywords	Flags
r	none	r
r	read = true	r
w	write = true	wct
w	truncate = true	wct
a	append = true	wca
r+	read = true, write = true	rw
w+	read = true, truncate = true	rwct
a+	read = true, append = true	rwca

There are two basic rules I extracted from the above table:

• write without read or append implies create and truncate
• truncate or append imply write and create

Unfortunately, this means that write and truncate can't be expressed with default values since their defaults depend on each other depending on which one is supplied. Moreover, another principle is that we don't want to override any explicitly set flag value. This means using nothing to mean "no value given".

[This would be a lot simpler if we were willing to spell open(write=true) as open(truncate=true) because there is a "truncate set" (wct) and an "append set" (wca) which are mostly orthogonal to the read set (r). These sets are combined in the + modes, except for r+ which we really want to write as read=true, write=true without the create or truncate flags implied by write by itself.]

Put together, this leads to the following implementation for getting flag values from keywords:

function open_keywords(;
    read     :: Union{Bool,Nothing} = nothing,
    write    :: Union{Bool,Nothing} = nothing,
    create   :: Union{Bool,Nothing} = nothing,
    truncate :: Union{Bool,Nothing} = nothing,
    append   :: Union{Bool,Nothing} = nothing,
)
    if write === true && read !== true && append !== true
        create   === nothing && (create   = true)
        truncate === nothing && (truncate = true)
    end

    if truncate === true || append === true
        write  === nothing && (write  = true)
        create === nothing && (create = true)
    end

    write    === nothing && (write    = false)
    read     === nothing && (read     = !write)
    create   === nothing && (create   = false)
    truncate === nothing && (truncate = false)
    append   === nothing && (append   = false)

    flags = Char[]
    read     && push!(flags, 'r')
    write    && push!(flags, 'w')
    create   && push!(flags, 'c')
    truncate && push!(flags, 't')
    append   && push!(flags, 'a')
    return String(flags)
end

This has the desired properties, but I have to confess that it's a bit more complex than I expected. Of course, when in doubt, just spell out flags – they will always be respected if given explicitly. I think the only other way to go is probably to just pick simple boolean defaults and say that people need to supply any flag that they don't want to assume its default value. Assuming read defaults to true and everything else false, that would lead to the following spelling table:

Mode	Keywords	Flags
r	none	r
r	read = true	r
w	read = false, write = true, create = true, truncate = true	wct
w	read = false, write = true, create = true, truncate = true	wct
a	read = false, write = true, create = true, append = true	wca
r+	read = true, write = true	rw
w+	read = true, write = true, create = true, truncate = true	rwct
a+	read = true, write = true, create = true, truncate = true	rwca

This is pretty verbose, but it does have the virtue of being straightforward. Of course, all of these spellings also work as desired with the DWIM open_keywords scheme, so you could always just forget that exists if you don't like it and just spell things out if you don't like the short versions.

[bramtayl]

My gut feeling is that we should just stick to the boolean keywords but I also don't understand well enough to know.

[JeffBezanson]

It may be that this interface just has too many degrees of freedom --- there are many combinations like read = true, create = true or read = true, truncate = true or append = true, write = false that don't make sense at face value.

[StefanKarpinski]

They're boolean keywords no matter what. The question is about defaults. Do we require people to supply any non-default boolean values for the read, write, create, truncate and append flags? In that case expressing a bog standard mode like w requires four keyword arguments:

open(file, read=false, write=true, create=true, truncate=true)

Or, do we make the defaults smarter (but more complicated) and allow them to just write this:

open(file, write=true)

The latter could potentially replace open(file, "w") while the former is clearly never going to.

[StefanKarpinski]

there are many combinations like read = true, create = true or read = true, truncate = true or append = true, write = false that don't make sense at face value.

Some of these do make some sense, although they're not very common. read + create would create an empty file if it doesn't exist and open it for reading, I believe. According to the man pages, truncate + read-only is undefined so that probably doesn't make sense. The above logic accordingly interprets read = true, truncate = true as w+ mode (i.e. read, write, create, truncate). append = true, write = false is a bit weird but it interprets as read, create, append which I believe would seek to read at the end of a file, creating it if it doesn't exist. I could imagine that being a useful mode for something similar to tail -f, reading new data appended to the file by another process.

[bramtayl]

What about

struct FileMode
    read::Bool
    write::Bool
    create::Bool
    truncate::Bool
    append::Bool
end

FileMode(; read = true, write = false, create = false, truncate = false, append = false) = 
    FileMode(read, write, create, truncate, append)

const read =  FileMode(true, false, false, false, false)
const read_plus = FileMode(true, true, false, false, false)
const write = FileMode(false, true, true, true, false)
const write_plus = FileMode(true, true, true, true, false)
const append = FileMode(false, true, true, false, true)
const append_plus = FileMode(false, true, true, false, true)

[StefanKarpinski]

How is that better than open(file, "r") and so on?

[bramtayl]

Oh maybe it isn't. Just throwing out ideas

[StefanKarpinski]

Worth a shot. I think we have a few options here:

1. Use the verbose, simple keyword approach but only consider it to be a more explicit way of writing the "long form" of open that we currently have taking a bunch of booleans. People will continue to use fopen specifiers like open(file, "w") almost all of the time.

2. Do something clever like what I've proposed above that lets people write open(file, write=true) to open a file in w mode or open(file, read=true, append=true) to open it in a+ mode and have a chance that people will adopt that over fopen specifiers.

3. Separate open into multiple functions/constructors with different defaults that can be overridden by keyword arguments. The trick would be figuring out what the verbs/nouns are.

[JeffBezanson]

In that case I vote for (2). Munging a few bools and nothings is no big deal, and it can actually be a bit simpler than what you have above since AFAIK we don't actually need to form that string.

[StefanKarpinski]

AFAIK we don't actually need to form that string.

Yes, that was just for testing and playing with it. I'll put up an alternate PR.

[JeffBezanson]

replaced by #25696