Use system MbedTLS #66

malmaud · 2016-08-14T22:53:31Z

No description provided.

tkelman · 2016-08-14T23:28:52Z

deps/build.jl

@@ -31,12 +31,28 @@ else
    srcsha = "8f25b6f156ae5081e91bcc58b02455926d9324035fe5f7028a6bb5bc0139a757"
 end

+# Use the version of MbedTLS that ships with Julia .5 and later
+
+julia_bin = Base.julia_cmd().exec[1]


just use JULIA_HOME, that's what this is derived from anyway - the representation of Cmd might change which could break this

codecov-io · 2016-08-15T02:50:41Z

Current coverage is 70.25% (diff: 100%)

Merging #66 into master will decrease coverage by 0.51%

@@             master        #66   diff @@
==========================================
  Files            10         10          
  Lines           390        390          
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits            276        274     -2   
- Misses          114        116     +2   
  Partials          0          0

Powered by Codecov. Last update 4a92329...bdc5685

malmaud · 2016-08-15T11:12:53Z

Travis is showing this error, even though things work locally for me. Confused ATM on what's going on.

LoadError: MethodError: no method matching generate_steps(::BinDeps.LibraryDependency, ::BinDeps.CustomPathBinaries, ::Dict{Symbol,Any})
Closest candidates are:
  generate_steps(::BinDeps.LibraryDependency, !Matched::BinDeps.AptGet, ::Any) at /Users/travis/.julia/v0.6/BinDeps/src/dependencies.jl:320
  generate_steps(::BinDeps.LibraryDependency, !Matched::BinDeps.Yum, ::Any) at /Users/travis/.julia/v0.6/BinDeps/src/dependencies.jl:331
  generate_steps(::BinDeps.LibraryDependency, !Matched::BinDeps.Pacman, ::Any) at /Users/travis/.julia/v0.6/BinDeps/src/dependencies.jl:343
  ...
while loading /Users/travis/.julia/v0.6/MbedTLS/deps/build.jl, in expression starting on line 83

tkelman · 2016-08-15T11:47:21Z

CustomPathBinaries is probably not widely tested with local paths. I'm not too familiar with where in BinDeps that code is.

malmaud · 2016-08-15T16:41:30Z

Alright. Realistically speaking, I'm probably not going to look into
patching BinDeps personally so this will remain open for the time being.

On Mon, Aug 15, 2016 at 6:47 AM Tony Kelman notifications@github.com
wrote:

CustomPathBinaries is probably not widely tested with local paths. I'm not
too familiar with where in BinDeps that code is.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#66 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA8SvZ6dEf6tpACjVUqllf7lcAbfB8MXks5qgFHJgaJpZM4JkAb7
.

tkelman · 2016-08-16T14:16:55Z

deps/build.jl

+# Use the version of MbedTLS that ships with Julia .5 and later
+
+provides(Binaries,
+    joinpath(dirname(JULIA_HOME), "..", "lib", "julia"),


One of either dirname or ".." are redundant, JULIA_HOME is already the bin directory. On Windows this should be just JULIA_HOME.

tkelman · 2016-08-16T14:38:18Z

deps/build.jl

+# For Unix
+provides(Binaries,
+    joinpath(JULIA_HOME, "..", "lib", "julia"),
+    mbed_all)


os = :Unix etc

malmaud · 2016-08-16T19:15:38Z

On Travis, it looks like MbedTLS is still being build from source on Linux and from Homebrew on OS X, even though the final file paths in deps.jl are correctly pointing the versions bundled with Julia. So that's a behavior of BinDeps that I don't really undestand.

PallHaraldsson · 2016-08-17T10:03:56Z

I do see "mbedtls (>= 2.2)" at Julia's README.md, while 2.2 is no longer listed as the supported version.

I made this #67 issue before seeing this one. You think Julia will better track security updates or just want to avoid duplicates? I did wander [for cURL], is that at least possible? Maybe better to track security updates (or not..)? [It's also a possibility that security updates for 0.4 might get neglected..]

malmaud · 2016-08-17T13:18:44Z

The versions of Julia without a bundled MbedTLS are failing on Travis: "LoadError: Provider BinDeps.Binaries failed to satisfy dependency libmbedtls".
I thought BinDeps would automatically try a different provider in that case? But I guess not and we need to check the Julia version before adding that binary provider?

tkelman · 2016-08-17T16:04:58Z

My impression was that it should also check several providers in a priority order and fall back to other options, but that might be broken?

malmaud · 2016-08-17T16:43:22Z

So that's one question, and the other is why the linux nightly is still building MbedTLS from source, and why Homebrew is still downloading an MbedTLS on the OS X nightly, even though in the latter two cases the final paths in deps.jl actually do point to the Julia-bundled binaries. Those all seem like issues with BinDeps.

staticfloat · 2016-09-06T20:49:29Z

deps/build.jl

+
+if is_unix()
+    provides(Binaries,
+        joinpath(JULIA_HOME, "..", "lib", "julia"),


Wow, this really should not be necessary. These libraries should be picked up automatically by the RPATH that is embedded in all Julia builds to find things like sys.so. Also, this will break if someone is running Julia with relative paths to libraries that aren't ../lib/julia, such as the Debian package where it's ../lib/x86_64-linux-gnu/julia, etc....

If you don't provide this path, do things break?

staticfloat · 2016-09-06T20:51:32Z

Can you embed a Bindeps.debug("MbedTLS") into your buildscript before trying to Pkg.build()? That will at least give us an idea of what BinDeps sees. You could also try running something like julia -e 'Libdl.dlopen("libmbedtls")' to see what the error message is that is causing Julia to think it doesn't have an libmbedtls laying around that it can use.

quinnj · 2016-11-18T15:10:26Z

With julia-provided shared library, do we really need BinDeps anymore? Is that just to provide fallbacks? I'm wondering if we can't just simplify the whole process here and say const mbedlib = "libmbedtls" and be done?

malmaud · 2016-11-18T15:33:35Z

Does that actually work right now?

On Nov 18, 2016, at 10:10 AM, Jacob Quinn notifications@github.com wrote:

With julia-provided shared library, do we really need BinDeps anymore? Is that just to provide fallbacks? I'm wondering if we can't just simplify the whole process here and say const mbedlib = "libmbedtls" and be done?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub #66 (comment), or mute the thread https://github.com/notifications/unsubscribe-auth/AA8SvV-1UaZW-Lc7-MnzNa0JDDgK4Decks5q_b_igaJpZM4JkAb7.

quinnj · 2016-11-18T15:49:05Z

              _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0 (2016-09-19 18:14 UTC)
 _/ |\__'_|_|_|\__'_|  |  Official http://julialang.org/ release
|__/                   |  x86_64-apple-darwin13.4.0

julia> data = Libc.malloc(32)
Ptr{Void} @0x00007f8aea45d330

julia> ccall((:mbedtls_pk_init, "libmbedcrypto"), Void, (Ptr{Void},), data)

julia>

I mean, 0.5-only I would imagine, but we could bump the Julia minimum.

malmaud · 2016-11-18T15:50:18Z

What do you think, @tkelman ? Any reason not to do this?

staticfloat · 2016-11-20T03:11:30Z

I'm in favor of using the julia-bundled libraries as much as we can, with the caveat that of course we always want to shrink Base as much as possible, so MbedTLS may go away some time in the future.

@quinnj to make sure that it's picking up the right libmbedcrypto, you should probably filter( x-> contains(x, "mbed"), Libdl.dllist()) after that. For my from-source build of master, things look good:

julia> filter( x-> contains(x, "mbed"), Libdl.dllist())
3-element Array{AbstractString,1}:
 "/Users/sabae/src/julia/usr/lib//libmbedtls.dylib"
 "/Users/sabae/src/julia/usr/lib//libmbedx509.dylib"
 "/Users/sabae/src/julia/usr/lib//libmbedcrypto.dylib"

quinnj · 2016-12-01T04:36:48Z

Implemented in #81

Use system MbedTLS

f6b57aa

malmaud mentioned this pull request Aug 14, 2016

Use mbedTLS binaries shipped with Julia 0.5 #54

Closed

tkelman reviewed Aug 14, 2016
View reviewed changes

Use JULIA_HOME for build

f61a399

malmaud mentioned this pull request Aug 15, 2016

Problem with CustomPathBinaries JuliaPackaging/BinDeps.jl#241

Open

tkelman reviewed Aug 16, 2016
View reviewed changes

Test BinDeps patch

bdc5685

tkelman reviewed Aug 16, 2016
View reviewed changes

deps/build.jl

# For Unix

provides(Binaries,

joinpath(JULIA_HOME, "..", "lib", "julia"),

mbed_all)

Copy link

Contributor

tkelman Aug 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os = :Unix etc

malmaud added 2 commits August 16, 2016 11:36

.

fc839ad

.

46369c3

PallHaraldsson mentioned this pull request Aug 17, 2016

Support Julia 0.5-pre JuliaWeb/LibCURL.jl#29

Merged

staticfloat reviewed Sep 6, 2016
View reviewed changes

tkelman mentioned this pull request Sep 7, 2016

Switch to MbedTLS JuliaLang/IJulia.jl#447

Merged

malmaud mentioned this pull request Oct 16, 2016

AptGet provider #74

Closed

quinnj closed this Dec 1, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use system MbedTLS #66

Use system MbedTLS #66

malmaud commented Aug 14, 2016

tkelman Aug 14, 2016

malmaud Aug 15, 2016

codecov-io commented Aug 15, 2016 •

edited

Loading

malmaud commented Aug 15, 2016 •

edited

Loading

tkelman commented Aug 15, 2016

malmaud commented Aug 15, 2016

tkelman Aug 16, 2016

tkelman Aug 16, 2016

malmaud commented Aug 16, 2016

PallHaraldsson commented Aug 17, 2016

malmaud commented Aug 17, 2016

tkelman commented Aug 17, 2016

malmaud commented Aug 17, 2016

staticfloat Sep 6, 2016

staticfloat commented Sep 6, 2016 •

edited

Loading

quinnj commented Nov 18, 2016

malmaud commented Nov 18, 2016

quinnj commented Nov 18, 2016

malmaud commented Nov 18, 2016

staticfloat commented Nov 20, 2016

quinnj commented Dec 1, 2016

Use system MbedTLS #66

Use system MbedTLS #66

Conversation

malmaud commented Aug 14, 2016

tkelman Aug 14, 2016

Choose a reason for hiding this comment

malmaud Aug 15, 2016

Choose a reason for hiding this comment

codecov-io commented Aug 15, 2016 • edited Loading

Current coverage is 70.25% (diff: 100%)

malmaud commented Aug 15, 2016 • edited Loading

tkelman commented Aug 15, 2016

malmaud commented Aug 15, 2016

tkelman Aug 16, 2016

Choose a reason for hiding this comment

tkelman Aug 16, 2016

Choose a reason for hiding this comment

malmaud commented Aug 16, 2016

PallHaraldsson commented Aug 17, 2016

malmaud commented Aug 17, 2016

tkelman commented Aug 17, 2016

malmaud commented Aug 17, 2016

staticfloat Sep 6, 2016

Choose a reason for hiding this comment

staticfloat commented Sep 6, 2016 • edited Loading

quinnj commented Nov 18, 2016

malmaud commented Nov 18, 2016

quinnj commented Nov 18, 2016

malmaud commented Nov 18, 2016

staticfloat commented Nov 20, 2016

quinnj commented Dec 1, 2016

codecov-io commented Aug 15, 2016 •

edited

Loading

malmaud commented Aug 15, 2016 •

edited

Loading

staticfloat commented Sep 6, 2016 •

edited

Loading