Second derivative of univariate functions: hessian() only takes arrays

[vlepori]

Unlike gradient, hessian only has an (f, x::AbstractArray) method, so it won't take Floats. One has to pass an array of length one, and use a wrapper (anonymous) function to extract the value. Example:

using Zygote

f(x) = (sin(x)/x)

g = x -> Zygote.gradient(f, x)
g(2.0) # OK

h = x -> Zygote.hessian(f, x)
h(2.0) # no method matching..

h = x -> Zygote.hessian(x-> f(x[1]), x)
h([2.0]) # OK

Btw, the f''() notation fails for my actual function with a Can't differentiate foreigncall expression error

[mcabbott]

This is a bug, the docstring says it should work.

But for now, you could consider writing ForwardDiff.derivative(x -> ForwardDiff.derivative(x -> sin(x)/x, x), pi/2) instead. Zygote's hessian uses ForwardDiff to do the second derivative, so your function will probably need to be something ForwardDiff can work with in any case.

[vlepori]

Thank you, that works, and without all the wrapping. For completeness, f'' fails when my functions has default arguments stored in a parameter Dict. But calling hessian or derivative works just fine. Example:

prms = Dict{Symbol,Float64}(:c => 1.0)

function myfun(x, c = prms[:c])
  3*x^2 + 2*x + c
end

ForwardDiff.derivative(x -> ForwardDiff.derivative(myfun, x), 3.0) # OK
Zygote.hessian(x -> myfun(x[]), [3.0]) # OK
myfun''(3.0) # ERROR: Can't differentiate foreigncall expression

[mcabbott]

Yes, unfortunately Zygote alone does not handle second derivatives very well. The rules it uses to work out gradients are often not written in such a way that they themselves are differentiable. Sometimes they work but it hasn't been something people focused on.

For this myfun, since you aren't differentiating with respect to c, you might be able to just hide it from Zygote, using ignore or @ignore.

For high-order derivatives of scalar functions, you may also look into TaylorSeries.jl.

[vlepori]

Great, thanks a lot for the explanations! I think ForwardDiff covers all my needs for the moment.