Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent eltype between map and broadcast for empty collections #20033

Closed
nalimilan opened this issue Jan 14, 2017 · 6 comments
Closed

Inconsistent eltype between map and broadcast for empty collections #20033

nalimilan opened this issue Jan 14, 2017 · 6 comments
Labels
domain:broadcast Applying a function over a collection domain:collections Data structures holding multiple items, e.g. sets kind:regression Regression in behavior compared to a previous version

Comments

@nalimilan
Copy link
Member

On Julia 0.5.0:

julia> h3(x::Float64...) = prod(x)
h3 (generic function with 1 method)

julia> map(h3, Int[])
0-element Array{Any,1}

julia> broadcast(h3, Int[])
0-element Array{Any,1}

On current master:

julia> h3(x::Float64...) = prod(x)
h3 (generic function with 1 method)

julia> map(h3, Int[])
0-element Array{Any,1}

julia> broadcast(h3, Int[])
0-element Array{Union{},1}

I guess returning Union{} is not intended here? Anyway, whatever element type we choose, I think we should be consistent. Cf. old discussion at #7258.

@nalimilan
Copy link
Member Author

I've opened a PR to make broadcast consistent with map: #20049.

Another inconsistency not covered by the PR:

julia> f(x::Float64) = 1
f (generic function with 1 method)

julia> map(f, Any[])
0-element Array{Any,1}

julia> broadcast(f, Any[])
0-element Array{Int64,1}

@kshyatt kshyatt added the domain:types and dispatch Types, subtyping and method dispatch label Jan 15, 2017
@martinholters
Copy link
Member

Also this:

julia> map((x) -> error(x), 1:0)
0-element Array{Union{},1}

julia> map((x) -> error(x), [])
0-element Array{Any,1}

Looks a bit like mapping over an empty array is the exception to the rule.

@JeffBezanson
Copy link
Sponsor Member

Also this:

julia> map((x::String)->error(), Int[])
0-element Array{Any,1}

julia> map(x->error(), Int[])
0-element Array{Union{},1}

It looks like return_type works differently when no method matches, vs. when a method matches and that method always throws.

@martinholters
Copy link
Member

It's not directly return_type; that actually works fine:

julia> Core.Inference.return_type((x) -> error(), Tuple{Int})
Union{}

julia> Core.Inference.return_type((x::String) -> error(), Tuple{Int})
Union{}

But the output element type is not determined with return_type(f, Tuple{eltype(A)}), it goes through some indirections, ending in something like return_type(first, Tuple{typeof(Generator(f,A))}). I think the underlying problem boils down to

julia> foo(f, x) = f(x)
foo (generic function with 1 method)

julia> @code_warntype foo(x->error(), 1)
Variables:
  #self#::#foo
  f::##39#40
  x::Int64

Body:
  begin 
      return (Base.throw)((Base.ErrorException)(((Core.getfield)((Core.getfield)(Base.Main, :Base)::Any, :string)::Any)()::Any)::ErrorException)::Union{}
  end::Union{}

julia> @code_warntype foo((x::String)->error(), 1)
Variables:
  #self#::#foo
  f::##41#42
  x::Int64

Body:
  begin 
      return (f::##41#42)(x::Int64)::Any
  end::Any

which is due to this. Interesting, but probably beyond the scope of this issue.

martinholters referenced this issue Feb 24, 2017
adds a world counter
and threads it everywhere
and tracks all backedges
add tests for some versions of #265
also updates the compile test to be world-aware
and add docs for method replacement / world age

note that MethodTables need to Base.serialized in a world-aware manner

and MethodError needs to be world aware

the age in MethodInstance is a copy of the value returned by typemap lookup
since we don't currently have a mechanism for accessing that value directly
plus, this seems easier anyways

handle MethodInstance age range updates correctly means
only updating them in-place when that doesn't potentially invalidate other
consumers that might also be holding a reference to it
and propagate this through the callers to type-inference

virtual edges are tracked via exact signature tuple types

inference is pure for backedges meaning that
we only add backedges that exist at the end of inference
and then add them to the correct MethodInstance object
which also make codegen caching world-aware

min/max validity of ml-matches results is threaded out of jl_matching_methods into inference
  eventually should probably switch this to use a Ref,
  or return the value, or something similarly more efficient than a
  singleton Array, but currently refpointer.jl is not part of Core.Inference

incremental deserialize is aware of world counter logic
by handling the follow cases:
 - "free-standing" methods (where age is at the default of 0), probably from a toplevel thunk or deserialize
 - garbage methods (really we just want to delete these, but, oh well). this is something inferred to have been replaced
 - applicable methods (this is the overwhelming majority of cases)
the logic between TypeMapEntry, Method, and MethodInstance all agree on these concepts

this uses Expr(:body) as an argument to eval to ensure that eval won't call expand,
which ensures that QuoteNode works correctly and the eval is reasonably fast
@nalimilan
Copy link
Member Author

@vtjnash Is there any chance this line could be changed, or should we find a workaround?

@andreasnoack andreasnoack added the kind:regression Regression in behavior compared to a previous version label Apr 3, 2017
@JeffBezanson JeffBezanson added domain:collections Data structures holding multiple items, e.g. sets domain:broadcast Applying a function over a collection and removed domain:types and dispatch Types, subtyping and method dispatch labels Apr 4, 2017
@nalimilan
Copy link
Member Author

Now consistent on current master:

# Example 1
julia>  h3(x::Float64...) = prod(x)
h3 (generic function with 1 method)

julia> map(h3, Int[])
0-element Array{Union{},1}

julia> broadcast(h3, Int[])
0-element Array{Union{},1}

# Example 2
julia> f(x::Float64) = 1
f (generic function with 1 method)

julia> map(f, Any[])
0-element Array{Int64,1}

julia> broadcast(f, Any[])
0-element Array{Int64,1}

# Example 3
julia> map((x) -> error(x), 1:0)
0-element Array{Union{},1}

julia> map((x) -> error(x), [])
0-element Array{Union{},1}

# Example 4
julia> map((x::String)->error(), Int[])
0-element Array{Union{},1}

julia> map(x->error(), Int[])
0-element Array{Union{},1}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:broadcast Applying a function over a collection domain:collections Data structures holding multiple items, e.g. sets kind:regression Regression in behavior compared to a previous version
Projects
None yet
Development

No branches or pull requests

5 participants