fall back to dynamic dispatch in union-splitting optimization #18046

JeffBezanson · 2016-08-16T04:35:08Z

This is a possible fix fo #17932. @dlfivefifty could you test this?

This error has been seen in several places and is worrying since it's very difficult to work around.

This should help #17932. If a function returns an unanticipated type that wasn't part of the `Union` seen by inference, do dynamic dispatch instead of erroring.

dlfivefifty · 2016-08-16T06:22:06Z

Unfortunately, it's not fixed:

Julia Version 0.6.0-dev.232
Commit b0cf591 (2016-08-16 04:29 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin15.6.0)
  CPU: Intel(R) Core(TM) i5-6600 CPU @ 3.30GHz
  WORD_SIZE: 64
  BLAS: libgfortblas
  LAPACK: liblapack
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, broadwell)
INFO: Recompiling stale cache file /Users/solver/.julia/lib/v0.6/ApproxFun.ji for module ApproxFun.
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:217
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:217
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:217
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:228
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:228
WARNING: Base.writemime is deprecated.
  likely near /Users/solver/.julia/v0.6/ApproxFun/src/Extras/show.jl:228
Multivariate tests
WARNING: the no-op `transpose` fallback is deprecated, and no more specific `transpose` method for ApproxFun.Fun{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Float64} exists. Consider `permutedims(x, [2, 1])` or writing a specific `transpose(x::ApproxFun.Fun{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Float64})` method if appropriate.
 in depwarn(::String, ::Symbol) at ./deprecated.jl:64
 in transpose(::ApproxFun.Fun{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Float64}) at ./deprecated.jl:771
 in *(::ApproxFun.ProductFun{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},ApproxFun.TensorSpace{Tuple{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}}},ApproxFun.RealBasis,2},Float64}, ::ApproxFun.Evaluation{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Bool,Float64}) at /Users/solver/.julia/v0.6/ApproxFun/src/PDE/PDE.jl:68
 in include_from_node1(::String) at ./loading.jl:426 (repeats 2 times)
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /Users/solver/.julia/v0.6/ApproxFun/test/MultivariateTest.jl, in expression starting on line 95
PDE tests
WARNING: sub is deprecated, use view instead.
 in depwarn(::String, ::Symbol) at ./deprecated.jl:64
 in sub(::BandedMatrices.BandedMatrix{Complex{Float64}}, ::UnitRange{Int64}, ::UnitRange{Int64}) at ./deprecated.jl:30
 in resizedata!(::ApproxFun.MutableOperator{Complex{Float64},ApproxFun.PlusOperator{Complex{Float64}},ApproxFun.SavedFunctional{Complex{Float64},M<:ApproxFun.Functional}}, ::Int64) at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/almostbanded/MutableOperator.jl:216
 in adaptiveqr!(::ApproxFun.MutableOperator{Complex{Float64},ApproxFun.PlusOperator{Complex{Float64}},ApproxFun.SavedFunctional{Complex{Float64},M<:ApproxFun.Functional}}, ::Array{Complex{Float64},1}, ::Float64, ::Int64) at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/adaptiveqr.jl:89
 in adaptiveqr(::Array{ApproxFun.Operator{Complex{Float64}},1}, ::Array{Complex{Float64},1}, ::Float64, ::Int64) at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/adaptiveqr.jl:81
 in #linsolve#136(::Float64, ::Int64, ::Function, ::Array{ApproxFun.Operator{Complex{Float64}},1}, ::Array{Complex{Float64},1}) at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/linsolve.jl:35
 in (::ApproxFun.#kw##linsolve)(::Array{Any,1}, ::ApproxFun.#linsolve, ::Array{ApproxFun.Operator{Complex{Float64}},1}, ::Array{Complex{Float64},1}) at ./<missing>:0
 in #linsolve#137(::Float64, ::Int64, ::Function, ::Array{ApproxFun.Operator{Complex{Float64}},1}, ::Array{Any,1}) at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/linsolve.jl:121
 in (::ApproxFun.#kw##linsolve)(::Array{Any,1}, ::ApproxFun.#linsolve, ::Array{ApproxFun.Operator{Complex{Float64}},1}, ::Array{Any,1}) at ./<missing>:0
 in exp(::ApproxFun.Fun{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Complex{Float64}}) at /Users/solver/.julia/v0.6/ApproxFun/src/Extras/specialfunctions.jl:392
 in include_from_node1(::String) at ./loading.jl:426 (repeats 2 times)
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /Users/solver/.julia/v0.6/ApproxFun/test/PDETest.jl, in expression starting on line 11
WARNING: super(T::DataType) is deprecated, use supertype(T) instead.
 in depwarn(::String, ::Symbol) at ./deprecated.jl:64
 in super(::DataType) at ./deprecated.jl:50
 in eltype at /Users/solver/.julia/v0.6/ApproxFun/src/Operators/Operator.jl:18 [inlined] (repeats 3 times)
 in kron(::Array{ApproxFun.Evaluation{ApproxFun.Ultraspherical{0,ApproxFun.Interval{Float64}},Bool,Float64},1}, ::UniformScaling{Int64}) at /Users/solver/.julia/v0.6/ApproxFun/src/PDE/KroneckerOperator.jl:295
 in dirichlet(::ApproxFun.ProductDomain{Tuple{ApproxFun.Interval{Float64},ApproxFun.Interval{Float64}},Float64,2}) at /Users/solver/.julia/v0.6/ApproxFun/src/PDE/PDE.jl:49
 in include_from_node1(::String) at ./loading.jl:426 (repeats 2 times)
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /Users/solver/.julia/v0.6/ApproxFun/test/PDETest.jl, in expression starting on line 16

signal (4): Illegal instruction: 4
while loading /Users/solver/.julia/v0.6/ApproxFun/test/PDETest.jl, in expression starting on line 16
copy! at ./abstractarray.jl:471
kron at /Users/solver/.julia/v0.6/ApproxFun/src/PDE/KroneckerOperator.jl:295
dirichlet at /Users/solver/.julia/v0.6/ApproxFun/src/PDE/PDE.jl:49
jl_call_method_internal at /Users/solver/Projects/julia5/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/solver/Projects/julia5/src/gf.c:1930
do_call at /Users/solver/Projects/julia5/src/interpreter.c:66
eval at /Users/solver/Projects/julia5/src/interpreter.c:205
do_call at /Users/solver/Projects/julia5/src/interpreter.c:65
eval at /Users/solver/Projects/julia5/src/interpreter.c:205
eval_body at /Users/solver/Projects/julia5/src/interpreter.c:481
jl_interpret_call at /Users/solver/Projects/julia5/src/interpreter.c:585
jl_toplevel_eval_flex at /Users/solver/Projects/julia5/src/toplevel.c:572
jl_parse_eval_all at /Users/solver/Projects/julia5/src/ast.c:717
jl_load at /Users/solver/Projects/julia5/src/toplevel.c:596 [inlined]
jl_load_ at /Users/solver/Projects/julia5/src/toplevel.c:605
include_from_node1 at ./loading.jl:426
jl_call_method_internal at /Users/solver/Projects/julia5/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/solver/Projects/julia5/src/gf.c:1930
do_call at /Users/solver/Projects/julia5/src/interpreter.c:66
eval at /Users/solver/Projects/julia5/src/interpreter.c:205
jl_toplevel_eval_flex at /Users/solver/Projects/julia5/src/toplevel.c:558
jl_parse_eval_all at /Users/solver/Projects/julia5/src/ast.c:717
jl_load at /Users/solver/Projects/julia5/src/toplevel.c:596 [inlined]
jl_load_ at /Users/solver/Projects/julia5/src/toplevel.c:605
include_from_node1 at ./loading.jl:426
jl_call_method_internal at /Users/solver/Projects/julia5/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/solver/Projects/julia5/src/gf.c:1930
process_options at ./client.jl:262
_start at ./client.jl:318
jl_call_method_internal at /Users/solver/Projects/julia5/src/./julia_internal.h:189 [inlined]
jl_apply_generic at /Users/solver/Projects/julia5/src/gf.c:1930
true_main at /Users/solver/Projects/julia5/usr/bin/julia (unknown line)
main at /Users/solver/Projects/julia5/usr/bin/julia (unknown line)
Allocations: 93930074 (Pool: 93926192; Big: 3882); GC: 164
Illegal instruction: 4

dlfivefifty · 2016-08-16T06:25:03Z

This is Pkg.status():

julia> Pkg.status()
1 required packages:
 - ApproxFun                     0.2.0-             bug-illinstruct (dirty)
25 additional packages:
 - BandedMatrices                0.0.4
 - BinDeps                       0.4.2
 - Calculus                      0.1.15
 - ColorTypes                    0.2.6
 - Colors                        0.6.6
 - Compat                        0.8.6
 - DualNumbers                   0.2.2
 - FastGaussQuadrature           0.0.3
 - FastTransforms                0.0.4
 - FixedPointNumbers             0.1.4
 - FixedSizeArrays               0.2.2
 - Iterators                     0.1.10
 - Measures                      0.0.3
 - NaNMath                       0.2.1
 - PlotUtils                     0.0.4
 - Plots                         0.9.0
 - RecipesBase                   0.0.6
 - Reexport                      0.0.3
 - Rmath                         0.1.2
 - SHA                           0.2.1
 - Showoff                       0.0.7
 - StatsBase                     0.9.0
 - StatsFuns                     0.3.0
 - ToeplitzMatrices              0.1.1
 - URIParser                     0.1.6

JeffBezanson · 2016-08-16T14:06:03Z

Thanks for checking. Looks like #17003, so this change might have just pushed the problem elsewhere.

dlfivefifty · 2016-08-16T19:58:29Z

Sorry, checked the wrong issue. I'll give this a try for #18046

dlfivefifty · 2016-08-16T22:02:52Z

#17932 is now fixed, thanks!!

dlfivefifty · 2016-08-16T22:03:21Z

Will this be backported to 0.5?

tkelman · 2016-08-22T11:51:41Z

@JeffBezanson I'm going to try this out to see if backporting it would fix Lora and ValueHistories, ref #18156. Can we merge this?

@nanosoldier runbenchmarks(ALL, vs = ":master")

This should help #17932. If a function returns an unanticipated type that wasn't part of the `Union` seen by inference, do dynamic dispatch instead of erroring. (cherry picked from commit b0cf591) ref #18046

nanosoldier · 2016-08-22T12:38:18Z

Something went wrong when running your job: RemoteException(3,CapturedException(NanosoldierError: failed to run benchmarks against primary commit: ErrorException("failed process: Process(make, ProcessExited(2)) [2]"),Any[( in pipeline_error at process.jl:612,1),( in run at process.jl:588,1),( in build_julia! at build.jl:54,1),( in execute_benchmarks! at BenchmarkJob.jl:251,1),( in run at BenchmarkJob.jl:178,1),( in #503 at multi.jl:1410,1),( in run_work_thunk at multi.jl:996,1),( in macro expansion at multi.jl:1410 [inlined],1),( in #502 at event.jl:46,1)]))
cc @jrevels

This should help #17932. If a function returns an unanticipated type that wasn't part of the `Union` seen by inference, do dynamic dispatch instead of erroring. (cherry picked from commit b0cf591) ref #18046

tkelman · 2016-08-22T15:35:15Z

The make failure on master is due to #18164. At least within #18156, this does look like it caused some benchmark regressions, but I think we can hopefully work on those, and it's less bad than having packages stop working for confusing reasons.

JeffBezanson · 2016-08-22T20:41:31Z

Yes, this change isn't ideal but we should probably go with it to prevent spurious breakage. There will probably be a lot more work on #265 in 0.6 anyway.

fall back to dynamic dispatch in union-splitting optimization

b0cf591

This should help #17932. If a function returns an unanticipated type that wasn't part of the `Union` seen by inference, do dynamic dispatch instead of erroring.

tkelman added the backport pending 0.5 label Aug 22, 2016

tkelman mentioned this pull request Aug 22, 2016

[release-0.5] backports for 0.5.0-rc3 #18156

Merged

JeffBezanson merged commit 969203a into master Aug 22, 2016

tkelman deleted the jb/17932 branch August 22, 2016 20:43

tkelman removed the backport pending 0.5 label Aug 22, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fall back to dynamic dispatch in union-splitting optimization #18046

fall back to dynamic dispatch in union-splitting optimization #18046

JeffBezanson commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016 •

edited

Loading

dlfivefifty commented Aug 16, 2016

JeffBezanson commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016 •

edited

Loading

dlfivefifty commented Aug 16, 2016

tkelman commented Aug 22, 2016

nanosoldier commented Aug 22, 2016

tkelman commented Aug 22, 2016

JeffBezanson commented Aug 22, 2016

fall back to dynamic dispatch in union-splitting optimization #18046

fall back to dynamic dispatch in union-splitting optimization #18046

Conversation

JeffBezanson commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016 • edited Loading

dlfivefifty commented Aug 16, 2016

JeffBezanson commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016

dlfivefifty commented Aug 16, 2016 • edited Loading

dlfivefifty commented Aug 16, 2016

tkelman commented Aug 22, 2016

nanosoldier commented Aug 22, 2016

tkelman commented Aug 22, 2016

JeffBezanson commented Aug 22, 2016

dlfivefifty commented Aug 16, 2016 •

edited

Loading

dlfivefifty commented Aug 16, 2016 •

edited

Loading