[Proposal] Support ! (not) on bool-returning delegates and method groups

[ashmind]

Problem

Consider these two situations:

var allowed = users.Where(IsAllowed);
var denied = users.Where(u => !IsAllowed(u));

While logic is symmetrical, the delegate usage is not.

Proposal

var allowed = users.Where(IsAllowed);
var denied = users.Where(!IsAllowed);

1. Allow operator ! to apply to bool-returning delegates and method groups. For simplicity, let's say only the ones that don't have ref or out parameters are allowed.

2. Result of applying ! to delegate of bool-returning type F is another delegate of type F, which passes all of its parameters to the original, then negates the original result.

3. When ! is applied to a method group, the method group is first processed as if ! wasn't present. After that it is similar to 2.

Concerns

Should we consider other operators? I wouldn't mind something like IsAllowed && !IsSuperUser to automatically generate u => IsAllowed(u) && !IsSuperUser(u), but it might be very hard to contextually resolve for method groups.

It feels significantly less common, so I am not including it in this proposal.

[canton7]

This is more complex than it appears.

The following two statements are not equivalent:

var auth = new Auth();
var allowed1 = users.Where(auth.IsAllowed);
var allowed2 = users.Where(x => auth.IsAllowed(x));

The first one creates a delegate which stores the current value of auth. The second one closes over the variable auth. This means that:

auth = null;
var array1 = allowed1.ToArray(); // Works fine
var array2 = allowed2.ToArray(); // NullReferenceException

SharpLab

There have been various attempts by various people to treat .Where(auth.IsAllowed) and .Where(x => auth.IsAllowed(x)) as equivalent (mainly to do with simplifying the latter to the former, as the latter involves an extra allocation), but they are not.

I suppose it would be possible to implement users.Where(!auth.IsAllowed) by lowering it to something similar to users.Where(x => !auth.IsAllowed(x)), but where auth isn't lifted: subsequent references to auth in the enclosing method would refer to the original local, not the field on the compiler-generated display class. That would add another allocation, but I don't suppose that would matter much.

---

I'm pretty sure I've seen a dup for this, but I can't find it...

[IS4Code]

You can make the compiler generate a static method for !auth.IsAllowed without extra allocations:

var predicate = (Func<User, bool>)Delegate.CreateDelegate(typeof(Func<User, bool>), auth, typeof(Program).GetMethod(nameof(AutoMethod)));
users.Where(predicate);

public static bool AutoMethod(Auth auth, User user)
{
    return !auth.IsAllowed(user);
}

It is not equivalent to the variable-capturing lambda, but it behaves like without !. Of course the compiler can simply emit ldftn without doing cumbersome reflection.

"Lifting" method groups this way for any combination of operators seems like a pretty cool idea as well:

users.Where(auth.GetUserLevel > 0);

A problem arises when local variables are used in the expression though, since that does require an allocation.

users.Where(auth.GetUserLevel > inputLevel);

It would be interesting if this kind of code could be used to create a lambda with a closure but without transforming locals to fields of the closure, i.e. capturing only their immediate values.

[CyrusNajmabadi]

We have considered this and very much so not like it. The lack of any syntax to distinguish a closure versus a non closure is basically a nonstarter

[jnm2]

What about for static method calls only, so that no closure is created? E.g. .Where(!string.IsNullOrEmpty) would be similar to:

.Where(static (p1) => !string.IsNullOrEmpty(p1))

[JustNrik]

You know you can make your own WhereNot extension method right?

public static IEnumerable<T> WhereNot<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
    if (source is null)
        throw new ArgumentNullException(nameof(source));

    if (predicate is null)
        throw new ArgumentNullException(nameof(predicate));

    return WhereNotIterator(source, predicate);
    
    IEnumerable<T> WhereNotIterator(IEnumerable<T> source, Func<T, bool> predicate)
    {
        foreach (var item in source)
        {
            if (!predicate(item))
                yield return item;
        }
    }
}

[iam3yal]

@JustNrik You don't really need all this, you can just return source.Where(x => !predicate(x)) but I'm not sure whether this is the point, I mean, you will have to introduce a new overload for each arity and while it's possible to create a library for this (which might be sensible but it's not just negation so it may not be that useful) it might be something the language can help us express better, maybe just for the common use case.

[ashmind]

We have considered this and very much so not like it. The lack of any syntax to distinguish a closure versus a non closure is basically a nonstarter

@CyrusNajmabadi Is it a closure in a normal sense? As mentioned by @IllidanS4 method group can produce Delegate with same Target, just different Method (that does not capture anything). In the delegate version original delegate itself can maybe be the Target? As below:

public static bool GeneratedIsNotOneArg(Func<User, bool> original, User arg) {
    return !original(arg);
}

[Thaina]

dotnet/runtime#26168

[r-pankevicius]

This is more complex than it appears.

The following two statements are not equivalent:

var auth = new Auth();
var allowed1 = users.Where(auth.IsAllowed);
var allowed2 = users.Where(x => auth.IsAllowed(x));

The first one creates a delegate which stores the current value of auth. The second one closes over the variable auth. This means that:

auth = null;
var array1 = allowed1.ToArray(); // Works fine
var array2 = allowed2.ToArray(); // NullReferenceException

SharpLab

There have been various attempts by various people to treat .Where(auth.IsAllowed) and .Where(x => auth.IsAllowed(x)) as equivalent (mainly to do with simplifying the latter to the former, as the latter involves an extra allocation), but they are not.

I suppose it would be possible to implement users.Where(!auth.IsAllowed) by lowering it to something similar to users.Where(x => !auth.IsAllowed(x)), but where auth isn't lifted: subsequent references to auth in the enclosing method would refer to the original local, not the field on the compiler-generated display class. That would add another allocation, but I don't suppose that would matter much.

I'm pretty sure I've seen a dup for this, but I can't find it...

Lazy evaluation has some hidden corners like in your example here (there are much more), but I will support @ashmind idea and will expand it.

[r-pankevicius]

These statements:

var allowed1 = users.Where(auth.IsAllowed);
var allowed2 = users.Where(x => auth.IsAllowed(x));

are not equivalent even now and they are both valid. But there is a trap: if you found a bug because the statement users.Where(auth.IsAllowed) should be the opposite check in some place you''ll automatically replace it with users.Where(u => auth.IsAllowed(u)) and will run into the issue if the scope variable changes.

So let me ignore non-equivalence as a valid counterargument and let me go even further.

Allow a sequence of bool returning functions accepting the same or convertible argument list to be grouped by all logical operators and automatically generate combined bool function for this expression.

So this statement:

bool mayExpectRainbow = MayBeRaining(forecast) && MayBeSunny(forecast);

could be written as:

bool mayExpectRainbow = (MayBeRaining && MayBeSunny)(forecast);

And in similar Linq example user would choose how to write the statement, both forms will be valid and will be not equivalent the same way:

bool daysWeMayExpectRainbow1 = forecasts.Where(MayBeRaining && MayBeSunny).Select(f => f.Day);
bool daysWeMayExpectRainbow2 = forecasts.Where(f => MayBeRaining(f) && MayBeSunny(f)).Select(f => f.Day);
bool daysWeMayExpectRainbow3 = forecasts.Where(f => (MayBeRaining && MayBeSunny)(f)).Select(f => f.Day);

And support not only && operator but other operators as well, e.g. unary not (as in original proposition), but now with chaining:

var notEmptyStrings = strings.Where(!string.IsNullOrEmpty);
var numericStrings = strings.Where(!string.IsNullOrEmpty && CanBeParsedAsInt);
bool daysWeMayNotExpectRainbow = forecasts.Where(MayBeRaining ^ MayBeSunny).Select(day); // weird one, incorrect one, but XOR is also logical operator :)

[r-pankevicius]

I made a test trying to achieve same effect with simple generic functions Not, And, etc.

However, when using them compiler requires generic type arguments to be specified and instead of:

Not(string.IsNullOrEmpty)

I had to use this syntax:

Not<string?>(string.IsNullOrEmpty)

[jnm2]

This is #129

[r-pankevicius]

This is #129

Thank for pointing it out.