torch_numopt.curvature_estimator module

Base class for curvature estimation class.

class CurvatureEstimator(ndim=2, uses_blocks=False)

Bases: ABC

Abstract base class for all curvature estimators.

A curvature estimator provides a matrix (or matrix-vector product) that approximates the Hessian of the objective. This can be the exact Hessian, Gauss-Newton, diagonal, or identity.

Subclasses must implement: - scaling_matrix: return the matrix (or block/scalar representation). - hvp: Hessian-vector product. - quadratic_form: pᵀ H p.

Methods

full_scaling_matrix(objective, params)	Return the curvature matrix as a single dense tensor.
hvp(objective, params, step_dir)	Compute the Hessian-vector product H * v.
quadratic_form(objective, params, step_dir)	Compute the quadratic form vᵀ H v.
reset()	Reset any internal state (intended to be used for quasi-Newton methods).
scaling_matrix(objective, params)	Obtain the curvature matrix in its native representation.
update()	Updates the parameters of the curvature estimator.

reset()

Reset any internal state (intended to be used for quasi-Newton methods). By default does nothing.

full_scaling_matrix(objective, params)

Return the curvature matrix as a single dense tensor.

Depending on the estimator’s ndim and uses_blocks, this method constructs a full matrix from the internal representation.

Return type:

Tensor

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

Returns:

torch.Tensor

Full square matrix of size (total_params, total_params).

abstract scaling_matrix(objective, params)

Obtain the curvature matrix in its native representation.

Return type:

Union[Iterable, Tensor]

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

Returns:

iterable or torch.Tensor

Representation of the matrix (scalar, vector, tuple of blocks, or full tensor).

abstract hvp(objective, params, step_dir)

Compute the Hessian-vector product H * v.

Return type:

Tuple[Tensor]

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

step_dirParams

Vector v (same structure as params).

Returns:

Params

Result of H * v.

abstract quadratic_form(objective, params, step_dir)

Compute the quadratic form vᵀ H v.

Return type:

Tensor

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

step_dirParams

Vector v.

Returns:

torch.Tensor

Scalar value vᵀ H v.

update()

Updates the parameters of the curvature estimator.