torch_numopt.algorithms.gradient_descent#

Gradient descent optimizers (first-order).

These optimizers use only the gradient (identity curvature). They are the simplest methods and serve as a baseline.

Classes

GradientDescent(params[, lr_init, lr_method])

Vanilla gradient descent with a fixed or adaptively initialized learning rate.

GradientDescentLS(params[, lr_init, ...])

Gradient descent with a line search to determine the step length.

GradientDescentLipschitz(params[, lr_init])

Gradient descent with a learning rate estimated from the Lipschitz constant.

GradientDescentTR(params[, lr_init, ...])

Gradient descent with a trust-region (Cauchy point) step.