2

I'm trying to apply automatic differentiation (ForwardDiff) to a function that contains an instance of find_zero (Roots) and am encountering an error that seems to relate to find_zero not accepting the ForwardDiff.Dual type.

Here's a (contrived) minimal working example that illustrates the issue:

using Distributions
using Roots
using StatsFuns
using ForwardDiff

function test_fun(θ::AbstractVector{T}) where T
    μ,σ,p = θ;
    z_star = find_zero(z -> logistic(z) - p, 0.0)
    return pdf(Normal(μ,σ),z_star)
end

test_fun([0.0,1.0,0.75])

ForwardDiff.gradient(test_fun,[0.0,1.0,0.75])

This results in the following error:

ERROR: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3})
Closest candidates are:
  Float64(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:200
  Float64(::T) where T<:Number at boot.jl:716
  Float64(::Irrational{:invsqrt2}) at irrationals.jl:189
  ...
Stacktrace:
 [1] convert(::Type{Float64}, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}) at ./number.jl:7
 [2] setproperty!(::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Symbol, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}) at ./Base.jl:34
 [3] update_state(::Roots.Secant, ::Roots.DerivativeFree{Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}}, ::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.UnivariateZeroOptions{Float64,Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:163
 [4] find_zero(::Roots.Secant, ::Roots.AlefeldPotraShi, ::Roots.DerivativeFree{Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}}, ::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.UnivariateZeroOptions{Float64,Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.NullTracks) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:868
 [5] find_zero(::Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}, ::Float64, ::Roots.Secant, ::Roots.AlefeldPotraShi; tracks::Roots.NullTracks, verbose::Bool, p::Nothing, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:689
 [6] #find_zero#36 at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:123 [inlined]
 [7] find_zero at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:120 [inlined]
 [8] #find_zero#5 at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:707 [inlined]
 [9] find_zero at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:707 [inlined]
 [10] test_fun at ./REPL[7856]:3 [inlined]
 [11] vector_mode_dual_eval at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/apiutils.jl:37 [inlined]
 [12] vector_mode_gradient(::typeof(test_fun), ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:106
 [13] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}, ::Val{true}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:19
 [14] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:17 (repeats 2 times)
 [15] top-level scope at REPL[7858]:1
 [16] run_repl(::REPL.AbstractREPL, ::Any) at /builddir/build/BUILD/julia/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288

I have limited experience using the FowardDiff package and am probably misunderstanding how the Dual type works, so I would really appreciate if someone knows how to solve this issue. Thanks so much!

A. Harris
  • 117
  • 3
  • https://juliadiff.org/ForwardDiff.jl/stable/user/limitations/... You probably have to implement the dual function yourself, but I have no idea how that is done with ForwardDiff.jl. – phipsgabler Jul 22 '21 at 14:16

1 Answers1

0
z_star = find_zero(z -> logistic(z) - p, 0.0)

You have a fixed initial condition which is non-dual. Make it dual.

z_star = find_zero(z -> logistic(z) - p, zero(eltype(θ))
Chris Rackauckas
  • 18,645
  • 3
  • 50
  • 81