OpenSSL_jll changes behavior of python dependencies loaded via PythonCall

[lassepe]

Cross posting from: avik-pal/Wandb.jl#39

TL;DR loading OpenSSL_jll before Wandb.jl causes Wandb.jl to error out with an SSL error. Loading them in the opposite order is fine.

Reproducer:

using OpenSSL_jll
using Wandb: Wandb

# trigger ssl error
function provoke_error(logger)
    logging_dict = Dict{String,Any}()
    logging_dict["samples"] = Wandb.Image(rand(2, 2))
    Wandb.log(logger, logging_dict; step=1)
end


logger = Wandb.WandbLogger(; project="test")
provoke_error(logger)

Error:

Click to expand error

ERROR: LoadError: Python: CommError: Can't connect to HTTPS URL because the SSL module is not available.
Python stacktrace:
 [1] _new_conn
   @ urllib3.connectionpool ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/urllib3/connectionpool.py:1059
 [2] _get_conn
   @ urllib3.connectionpool ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/urllib3/connectionpool.py:294
 [3] urlopen
   @ urllib3.connectionpool ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/urllib3/connectionpool.py:768
 [4] send
   @ requests.adapters ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/requests/adapters.py:667
 [5] send
   @ requests.sessions ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/requests/sessions.py:703
 [6] request
   @ requests.sessions ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/requests/sessions.py:589
 [7] post
   @ requests.sessions ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/requests/sessions.py:637
 [8] execute
   @ wandb.sdk.lib.gql_request ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/lib/gql_request.py:58
 [9] _get_result
   @ wandb_gql.client ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py:60
 [10] execute
   @ wandb_gql.client ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/vendor/gql-0.2.0/wandb_gql/client.py:52
 [11] execute
   @ wandb.sdk.internal.internal_api ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/internal/internal_api.py:345
 [12] __call__
   @ wandb.sdk.lib.retry ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/lib/retry.py:131
 [13] gql
   @ wandb.sdk.internal.internal_api ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/internal/internal_api.py:317
 [14] server_info_introspection
   @ wandb.sdk.internal.internal_api ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/internal/internal_api.py:557
 [15] wrapper
   @ wandb.apis.normalize ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/apis/normalize.py:41
 [16] wrapper
   @ wandb.apis.normalize ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/apis/normalize.py:87
 [17] max_cli_version
   @ wandb.sdk.internal.internal_api ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/internal/internal_api.py:870
 [18] wrapper
   @ wandb.apis.normalize ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/apis/normalize.py:41
 [19] wrapper
   @ wandb.apis.normalize ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/apis/normalize.py:73
 [20] max_cli_version
   @ wandb.apis.internal ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/apis/internal.py:82
 [21] _get_max_cli_version
   @ wandb.util ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/util.py:1710
 [22] _server_accepts_artifact_path
   @ wandb.sdk.data_types.image ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/data_types/image.py:57
 [23] bind_to_run
   @ wandb.sdk.data_types.image ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/data_types/image.py:402
 [24] val_to_json
   @ wandb.sdk.data_types.utils ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/data_types/utils.py:164
 [25] history_dict_to_json
   @ wandb.sdk.data_types.utils ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/data_types/utils.py:52
 [26] publish_partial_history
   @ wandb.sdk.interface.interface ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/interface/interface.py:600
 [27] _partial_history_callback
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:1513
 [28] _log
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:1641
 [29] log
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:1877
 [30] wrapper
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:390
 [31] wrapper_fn
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:400
 [32] wrapper
   @ wandb.sdk.wandb_run ~/worktree/BugReports/Wandb.jl-load-order/.CondaPkg/env/lib/python3.12/site-packages/wandb/sdk/wandb_run.py:449
Stacktrace:
  [1] pythrow()
    @ PythonCall.Core ~/.julia/packages/PythonCall/S5MOg/src/Core/err.jl:92
  [2] errcheck
    @ ~/.julia/packages/PythonCall/S5MOg/src/Core/err.jl:10 [inlined]
  [3] pycallargs
    @ ~/.julia/packages/PythonCall/S5MOg/src/Core/builtins.jl:213 [inlined]
  [4] pycall(f::PythonCall.Core.Py, args::PythonCall.Core.Py; kwargs::@Kwargs{step::Int64})
    @ PythonCall.Core ~/.julia/packages/PythonCall/S5MOg/src/Core/builtins.jl:224
  [5] pycall
    @ ~/.julia/packages/PythonCall/S5MOg/src/Core/builtins.jl:220 [inlined]
  [6] Py
    @ ~/.julia/packages/PythonCall/S5MOg/src/Core/Py.jl:339 [inlined]
  [7] log(lg::Wandb.WandbLogger, logs::Dict{String, Any}; kwargs::@Kwargs{step::Int64})
    @ Wandb ~/worktree/BugReports/Wandb.jl-load-order/dev/Wandb/src/main.jl:69
  [8] log
    @ ~/worktree/BugReports/Wandb.jl-load-order/dev/Wandb/src/main.jl:69 [inlined]
  [9] provoke_error(logger::Wandb.WandbLogger)
    @ Main ~/worktree/BugReports/Wandb.jl-load-order/main.jl:8
 [10] top-level scope
    @ ~/worktree/BugReports/Wandb.jl-load-order/main.jl:13

[giordano]

When you mix libraries coming from different sources all odds are off.

[lassepe]

But is it expected that libs loaded via _jll are visible inside PythonCall dependencies? Is there no way to isolate those two environments?

[lassepe]

To be clear, I am not sure whether this "isolation" would be the _jll's responsibility or there needs to be a change on the side of PythonCall.jl --- just trying to build an understanding of where the issue needs to be filed.

[giordano]

But is it expected that libs loaded via _jll are visible inside PythonCall dependencies?

Yes, of course, it's all in the same process.

Is there no way to isolate those two environments?

There are no "two environments", there's a single process. That's how you move data seamlessly between "Julia code" and "python code".

[lassepe]

I see. I never realized that the python dependency would adopt the exact same process but it makes a lot of sense. I am still surprised that there's no way of name-spacing the libraries somehow, e.g. using some mechanism like dlmopen. If that is infeasible, please feel free to close this issue! Thank you for your swift response, @giordano!

[giordano]

e.g. using some mechanism like dlmopen

dlmopen was suggested at #3801 (comment), but (a) it's glibc-specific so it doesn't solve anything on all other platforms (b) it has its own problems.