Make the test function be an extension?

[ChrisRackauckas]

No other libraries depend on Test. It should probably be kept as an extension so that it only shows up by default as a test dependency.

[hameye]

@ChrisRackauckas Hi, sorry if this message should be posted elsewhere.
It seems this upgrade induces a cyclic import and leads to an error in my environment. It seems to be because I am using GeoStatsBase v0.31. Please find the corresponding error.

Precompiling project...
  ✗ GeoStatsBase
  136 dependencies successfully precompiled in 37 seconds. 8 already precompiled.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
  3 dependencies had warnings during precompilation:
┌ CompositionsBaseInverseFunctionsExt [62a8cebd-3339-5698-a5de-13c35b75e11f]
│  ┌ Warning: Module InverseFunctionsTestExt with build ID ffffffff-ffff-ffff-0000-0a1f641c096a is missing from the cache.
│  │ This may mean InverseFunctionsTestExt [05f3ecc3-30a3-5b98-a414-5bae29b80c93] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension InverseFunctionsTestExt of InverseFunctions, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1797
│  │      [2] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1660
│  │      [3] _require_prelocked(uuidkey::Base.PkgId)
│  │        @ Base ./loading.jl:1658
│  │      [4] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1255
│  │      [5] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1290
│  │      [6] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1124
│  │      [7] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1667
│  │      [8] macro expansion
│  │        @ ./loading.jl:1648 [inlined]
│  │      [9] macro expansion
│  │        @ ./lock.jl:267 [inlined]
│  │     [10] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1611
│  │     [11] include
│  │        @ ./Base.jl:457 [inlined]
│  │     [12] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2049
│  │     [13] top-level scope
│  │        @ stdin:3
│  │     [14] eval
│  │        @ ./boot.jl:370 [inlined]
│  │     [15] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:1903
│  │     [16] include_string
│  │        @ ./loading.jl:1913 [inlined]
│  │     [17] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:305
│  │     [18] _start()
│  │        @ Base ./client.jl:522
│  └ @ Base loading.jl:1261
└  
┌ InverseFunctionsTestExt [05f3ecc3-30a3-5b98-a414-5bae29b80c93]
│  ┌ Warning: Module InverseFunctionsTestExt with build ID ffffffff-ffff-ffff-0000-0a1f30ff07e0 is missing from the cache.
│  │ This may mean InverseFunctionsTestExt [05f3ecc3-30a3-5b98-a414-5bae29b80c93] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension InverseFunctionsTestExt of InverseFunctions, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1797
│  │      [2] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1660
│  │      [3] _require_prelocked(uuidkey::Base.PkgId)
│  │        @ Base ./loading.jl:1658
│  │      [4] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1255
│  │      [5] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1290
│  │      [6] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1124
│  │      [7] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1667
│  │      [8] macro expansion
│  │        @ ./loading.jl:1648 [inlined]
│  │      [9] macro expansion
│  │        @ ./lock.jl:267 [inlined]
│  │     [10] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1611
│  │     [11] include
│  │        @ ./Base.jl:457 [inlined]
│  │     [12] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2049
│  │     [13] top-level scope
│  │        @ stdin:3
│  │     [14] eval
│  │        @ ./boot.jl:370 [inlined]
│  │     [15] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:1903
│  │     [16] include_string
│  │        @ ./loading.jl:1913 [inlined]
│  │     [17] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:305
│  │     [18] _start()
│  │        @ Base ./client.jl:522
│  └ @ Base loading.jl:1261
└  
┌ InverseFunctionsDatesExt [33677a4c-2dac-51eb-93f3-7ff1c38acef2]
│  ┌ Warning: Module InverseFunctionsDatesExt with build ID ffffffff-ffff-ffff-0000-0a1f2b263cb9 is missing from the cache.
│  │ This may mean InverseFunctionsDatesExt [33677a4c-2dac-51eb-93f3-7ff1c38acef2] does not support precompilation but is imported by a module that does.
│  └ @ Base loading.jl:1793
│  ┌ Error: Error during loading of extension InverseFunctionsDatesExt of InverseFunctions, use `Base.retry_load_extensions()` to retry.
│  │   exception =
│  │    1-element ExceptionStack:
│  │    Declaring __precompile__(false) is not allowed in files that are being precompiled.
│  │    Stacktrace:
│  │      [1] _require(pkg::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1797
│  │      [2] _require_prelocked(uuidkey::Base.PkgId, env::Nothing)
│  │        @ Base ./loading.jl:1660
│  │      [3] _require_prelocked(uuidkey::Base.PkgId)
│  │        @ Base ./loading.jl:1658
│  │      [4] run_extension_callbacks(extid::Base.ExtensionId)
│  │        @ Base ./loading.jl:1255
│  │      [5] run_extension_callbacks(pkgid::Base.PkgId)
│  │        @ Base ./loading.jl:1290
│  │      [6] run_package_callbacks(modkey::Base.PkgId)
│  │        @ Base ./loading.jl:1124
│  │      [7] _require_prelocked(uuidkey::Base.PkgId, env::String)
│  │        @ Base ./loading.jl:1667
│  │      [8] macro expansion
│  │        @ ./loading.jl:1648 [inlined]
│  │      [9] macro expansion
│  │        @ ./lock.jl:267 [inlined]
│  │     [10] require(into::Module, mod::Symbol)
│  │        @ Base ./loading.jl:1611
│  │     [11] include
│  │        @ ./Base.jl:457 [inlined]
│  │     [12] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::String)
│  │        @ Base ./loading.jl:2049
│  │     [13] top-level scope
│  │        @ stdin:3
│  │     [14] eval
│  │        @ ./boot.jl:370 [inlined]
│  │     [15] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
│  │        @ Base ./loading.jl:1903
│  │     [16] include_string
│  │        @ ./loading.jl:1913 [inlined]
│  │     [17] exec_options(opts::Base.JLOptions)
│  │        @ Base ./client.jl:305
│  │     [18] _start()
│  │        @ Base ./client.jl:522
│  └ @ Base loading.jl:1261
└  

To reproduce the issue, an empty environment with the following Project.toml

[deps]
GeoStatsBase = "323cb8eb-fbf6-51c0-afd0-f8fba70507b2"

[compat]
GeoStatsBase = "0.31"

Thank you for your answers

[devmotion]

The problem is the same as described in #52 (comment):

(jl_P3hAQu) pkg> add GeoStatsBase#master

(jl_P3hAQu) pkg> why Test
  GeoStatsBase → GeoTables → CoordRefSystems → Zygote → ForwardDiff → CommonSubexpressions → Test
  GeoStatsBase → GeoTables → Meshes → CoordRefSystems → Zygote → ForwardDiff → CommonSubexpressions → Test
  GeoStatsBase → Meshes → CoordRefSystems → Zygote → ForwardDiff → CommonSubexpressions → Test
  GeoStatsBase → Optim → ForwardDiff → CommonSubexpressions → Test
  GeoStatsBase → Optim → LineSearches → NLSolversBase → ForwardDiff → CommonSubexpressions → Test
  GeoStatsBase → Optim → NLSolversBase → ForwardDiff → CommonSubexpressions → Test

GeoStatsBase indirectly depends on Test. This should be fixed by a new release of CommonSubExpressions: rdeits/CommonSubexpressions.jl#27

[oschulz]

Is this a general problem that might occur again, or will this go away anyway with the next 1.10.x release?

[devmotion]

Since JuliaLang/julia#52511 is still open, I think there's no fix in Julia master that would resolve this problem. My impression is that for the time being it's best to track down indirect dependencies that cause such circular dependencies and resolve it by fixing those. I wonder if as a workaround one could break the circular loading by making Test a direct dependency of GeoStatsBase (of course, that's not the right fix, I'm just curious about the Pkg behaviour).

[ChrisRackauckas]

This is a case where everything works well if there are no bad actors, so it needs to be eliminated from the full dependency stack. If no one has a direct on Test then there cannot be a circular to it.

[oschulz]

If no one has a direct on Test then there cannot be a circular to it.

Alright, let's go lobby other packages then! :-)

[devmotion]

I can't reproduce the warnings anymore with the latest release of CommonSubExpressions.