-1

In Linear Regression, we have formulas to calculate the slope and intercept, to find the best fit line; then why do we need to use Gradient Descent for calculating the optimum slope & intercept, which we already get by given formulas?

desertnaut
  • 57,590
  • 26
  • 140
  • 166
Gaurav Kamble
  • 21
  • 2

1 Answers1

1

I guess you are referring to the closed form solution of the linear regression.

And yes - you can totally fine use it for that purpose. However, this only works as long as you have all your data being able to fit into memory, which you wouldn't necessarily need when using gradient descend: Since Gradient Descend is an iterative process you can swap in and out some data making it more appropriate for big data.

See the Cross Validated thread Solving for regression parameters in closed-form vs gradient descent for more details.

Ric Hard
  • 599
  • 4
  • 12
  • I want to know whether Linear Regression uses Gradient Descent or simple formulas for slope & intercept, to calculate the best fit line for the Predictions. – Gaurav Kamble Mar 24 '21 at 17:25
  • 2
    @GauravKamble the answer is trying to say that in the "small data" case you use the "simple formulas" you refer to; however, these formulas are not applicable in the case where the data do not fit into memory ("big data"), in which case GD is used instead; something you would have figured out already if you cared to consult the linked CV thread, as suggested. – desertnaut Mar 24 '21 at 21:15
  • Reading your comment, I think your misconception is / was considering linear regression as one stand-alone algorithm. However, linear regression is just a specific **model** and both the closed form solution and gradient descend are algorithms to **solve / optimize** the model for a dataset. – Ric Hard Mar 25 '21 at 10:57