Using Deriv package for derivative wrt vector

Question

I am exploring autodiff, and I would like to use Deriv for computing a derivative of a function wrt to a vector. I write

library(numDeriv)
library(Deriv)
h = function(x) c(1,2)%*%x
grad(h,c(1,2)) #ok
#[1] 1 2
dh=Deriv(h,x='x')
#Error in c(1, 2) %*% 1 : non-conformable arguments
dh(c(1,2))

Does anyone have a good way to do this?

From help(Deriv), it seems like one should be able to let the argument be a vector

here is a side effect with vector length. E.g. in Deriv(~a+bx, c("a", "b")) the result is c(a = 1, b = x). To avoid the difference in lengths of a and b components (when x is a vector), one can use an optional parameter combine Deriv(~a+bx, c("a", "b"), combine="cbind") which gives cbind(a = 1, b = x) producing a two column matrix which is probably the desired result here.

I would like to avoid making each of the vector components a different argument to the function.

For example numDeriv above lets us easily take a derivative wrt vector x

The problem isn't with Deriv its that you can't pass a single numeric vector to matrix multiplication. i.e. h(1) will give you the error you see but h(c(1,4)) will not. — Henry Holm, Nov 07 '21 at 20:53
@BenBolker - Thank you - I have looked into it a bit, but I wanted to check first whether `Deriv` supports autodiff wrt a vector, and if I was just making a mistake. Maybe I will go in that direction — Sam Weisenthal, Nov 07 '21 at 20:59
@HenryHolm thanks - would you be able to provide an example using Deriv? — Sam Weisenthal, Nov 07 '21 at 21:00

score 2 · Answer 1 · answered Nov 07 '21 at 20:55

2

This is an answer; The to packages handles multiple dimensions differently.


library(numDeriv)
library(Deriv)
h = function(x,y) c(1,2) %*% c(x,y)
grad(\(x) h(x[1], x[2]),c(1,2))
dh = Deriv(h)
dh(c(1,2))

answered Nov 07 '21 at 20:55

Mossa

1,656
12
16

thanks - I had wanted to avoid making the components separate args because then if I have a 100 dimensional vector it would require 100 inputs – Sam Weisenthal Nov 07 '21 at 21:04
maybe I am missing something about passing though args .. eg is there like a `...` operator I can use to avoid this.. – Sam Weisenthal Nov 07 '21 at 21:06
still it seems like it would be ideal if Deriv just could take a vector argument – Sam Weisenthal Nov 07 '21 at 21:06
What do you mean by many inputs? Multiple dimensions or multiple points? because you could make a function `base::Vectorize`, and thus make it take multiple points of a certain dimension. – Mossa Nov 07 '21 at 21:12
`h = function(x,y)` has 2 inputs whereas `h = function(x)` has one – Sam Weisenthal Nov 07 '21 at 21:14
Well, put up an example of multiple argument function you want to differentiate... – Mossa Nov 07 '21 at 21:15
I wanted to stay with `h = function(x) c(1,2)%*%x` where `x` is a vector rather than `h = function(x,y) c(1,2)%*%c(x,y)` where `x,y` are scalars. – Sam Weisenthal Nov 07 '21 at 21:19
1

Maybe the `Vectorize` function you mention might allow me to avoid this though.. – Sam Weisenthal Nov 07 '21 at 21:20
So if I had function `h=function(x) 1:1000%*%x` how would I use the method above – Sam Weisenthal Nov 07 '21 at 21:21
2

I cannot figure that out. Might be impossible. Sorry – Mossa Nov 07 '21 at 21:27

score 1 · Answer 2 · answered Nov 08 '21 at 02:03

Here is a solution using not Deriv but madness, a really neat package.

We basically create an object that is the thing we would like to take the derivative with respect to (in this case x), and then as we apply functions to that object, the derivatives are collected.

We get the evaluated derivative using this function, as we do with grad in numDeriv.

library(madness)

h = function(x){t(x)%*%matrix(c(2,1),nrow=2,ncol=1)}
x=matrix(c(1,1),nrow=2,ncol=1)

gd=function(h,x){
x=madness(val=x)
z=h(x)
attr(z,"dvdx")
}
gd(h,x)
#     [,1] [,2]
#[1,]    2    1

score 1 · Answer 3 · answered Nov 09 '21 at 22:29

This feels really kludgy:

library(Deriv)

sz <- 100
f.vec <- function(x) 1:sz%*%x
xs <- paste0("x", 1:sz, collapse = ",")
h <- eval(parse(text = paste0("h <- function(", xs, ") f.vec(c(", xs, "))")))
dh <- Deriv(h)
dh(1:sz)
#>   x1   x2   x3   x4   x5   x6   x7   x8   x9  x10  x11  x12  x13  x14  x15  x16 
#>    1    2    3    4    5    6    7    8    9   10   11   12   13   14   15   16 
#>  x17  x18  x19  x20  x21  x22  x23  x24  x25  x26  x27  x28  x29  x30  x31  x32 
#>   17   18   19   20   21   22   23   24   25   26   27   28   29   30   31   32 
#>  x33  x34  x35  x36  x37  x38  x39  x40  x41  x42  x43  x44  x45  x46  x47  x48 
#>   33   34   35   36   37   38   39   40   41   42   43   44   45   46   47   48 
#>  x49  x50  x51  x52  x53  x54  x55  x56  x57  x58  x59  x60  x61  x62  x63  x64 
#>   49   50   51   52   53   54   55   56   57   58   59   60   61   62   63   64 
#>  x65  x66  x67  x68  x69  x70  x71  x72  x73  x74  x75  x76  x77  x78  x79  x80 
#>   65   66   67   68   69   70   71   72   73   74   75   76   77   78   79   80 
#>  x81  x82  x83  x84  x85  x86  x87  x88  x89  x90  x91  x92  x93  x94  x95  x96 
#>   81   82   83   84   85   86   87   88   89   90   91   92   93   94   95   96 
#>  x97  x98  x99 x100 
#>   97   98   99  100

The evaluation time for Deriv seems to increase roughly quadratically with the number of parameters:

varDeriv <- function(sz) {
  system.time({
    f.vec <- function(x) 1:sz%*%x
    xs <- paste0("x", 1:sz, collapse = ",")
    h <- eval(parse(text = paste0("h <- function(", xs, ") f.vec(c(", xs, "))")))
    dh <- Deriv(h)
    dh(1:sz)
  })
}

Vectorize(varDeriv)(seq(100, 500, by = 100))
#>             [,1]  [,2]  [,3]   [,4]   [,5]
#> user.self  0.712 2.700 6.498 11.782 20.854
#> sys.self   0.015 0.011 0.020  0.055  0.068
#> elapsed    0.727 2.710 6.518 11.839 20.922
#> user.child 0.000 0.000 0.000  0.000  0.000
#> sys.child  0.000 0.000 0.000  0.000  0.000

Using Deriv package for derivative wrt vector

3 Answers3

Linked