torch_numopt.algorithms.adahessian

AdaHessian optimizer (diagonal Hessian with momentum).

This module implements the AdaHessian algorithm, which combines the adaptive learning rate mechanism of Adam with a diagonal Hessian approximation computed via Hutchinson’s method. It maintains moving averages of both the gradient and the Hessian diagonal, and uses them to compute a preconditioned step direction.

Classes

AdaHessian(params[, lr_init, lr_method, ...])	AdaHessian optimizer (diagonal Hessian with momentum).
AdaHessianLS(params[, lr_init, lr_method, ...])	AdaHessian with line search.
AdaHessianMixin(*args[, beta1, beta2, k, eps])	Mixin that implements the AdaHessian algorithm.