0

I have a question about automatic differentiation and especially in Pytorch since i am using this library.I have seen for instance automatic differentiation give the partial derivatives of an expression with respect to a variable.

However,as far as what i have seen ,the result is always given at a specific point,meaning it is a tensor with numerical values.My question is the following: let's say we define a function of two variables: f(x,y)= x² + y² .

Is Pytorch able to return a function which corresponds to the partial derivative of f with respect to x or y? That is to return the following definitions:

> def partial_f_x:

      return 2*x

  def partial_f_y:
    
      return 2*y

Because even though the function f here is rather simple,it would be interesting if Pytorch could give us a formula(depending on the different variables) of the derivatives,instead of giving a numerical value at a given point,because in that case we do not know the expression of the derivatives.

So if I summarize: is Pytorch able to return formulas for derivatives of complicated functions? or does it just return a tensor with numerical values for the derivative at a given point?

Thank you so much!

sosamm
  • 39
  • 1
  • 6
  • 1
    I found these links which say something about how Pytorch autograd works. https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html https://pytorch.org/docs/stable/notes/autograd.html Maybe you are already aware of these. I wasn't able to tell from that whether you can get autograd to actually return a symbolic form of the gradient; it appears the answer is no, but I might very well be missing something. – Robert Dodier Feb 22 '21 at 21:39

1 Answers1

1

That is not how pytorch is doing to get the derivative. Most (probably all) of the computational packages are using approximation methods to get the derivative value rather than deriving the derivative function, so they don't care what is the derivative in mathematical terms.

If you are looking for something like that, you can try to use the sympy library for symbolic mathematics. Here's an example:

import sympy as sym

x = sym.Symbol('x')
y = sym.Symbol('y')

sym.diff(x**2 + y**2, x, 1)
# => 2*x

sym.diff(x**2 + y**2, y, 1)
# => 2*y

Then to evaluate, you can simply substitute in the values you want to use for the variable(s):

dfdx.subs(y,1)
# => 2
TYZ
  • 8,466
  • 5
  • 29
  • 60
  • 1
    "Most (probably all) of the computational packages are using approximation methods to get the derivative value rather than deriving the derivative function" -- that doesn't appear to be correct in reference to Pytorch, according to the links which I posted in response to OP above. – Robert Dodier Feb 22 '21 at 21:40
  • Thank you so much for your answer,but what if we have data points (x , f(x) ) representing points and their values by the function f,and we make a neural network to find which function it approximates using Pytorch?Can we actually get a function instead of numerical values at different points?If we cannot,what is then the purpose of learning functions? Thanks – sosamm Feb 23 '21 at 07:46
  • @RobertDodier Thanks for the reference, I guess I wasn't correct in that sense as I was digging very deep into the mechanics of pytorch implementation details. – TYZ Feb 23 '21 at 14:14
  • @sosam What do you mean by "learning functions"? Are you referring to objective function or loss function? – TYZ Feb 23 '21 at 14:18