torch_numopt.curvature.naive_identity module

Identity curvature estimator (no curvature).

This estimator treats the Hessian as the identity matrix, i.e., it ignores second-order information and effectively performs gradient descent.

class NaiveIdentityCalculator

Bases: CurvatureEstimator

Curvature estimator that always returns the identity matrix.

The scaling matrix is 1 (scalar), the Hessian-vector product is the vector itself, and the quadratic form is the squared norm.

Methods

full_scaling_matrix(objective, params)	Return the curvature matrix as a single dense tensor.
hvp(objective, params, step_dir)	Compute the Hessian-vector product H * v.
quadratic_form(objective, params, step_dir)	Compute the quadratic form vᵀ H v.
reset()	Reset any internal state (intended to be used for quasi-Newton methods).
scaling_matrix(objective, params)	Obtain the curvature matrix in its native representation.
update()	Updates the parameters of the curvature estimator.

scaling_matrix(objective, params)

Obtain the curvature matrix in its native representation.

Return type:

float

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

Returns:

iterable or torch.Tensor

Representation of the matrix (scalar, vector, tuple of blocks, or full tensor).

hvp(objective, params, step_dir)

Compute the Hessian-vector product H * v.

Return type:

Iterable[Tensor]

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

step_dirParams

Vector v (same structure as params).

Returns:

Params

Result of H * v.

quadratic_form(objective, params, step_dir)

Compute the quadratic form vᵀ H v.

Return type:

Tensor

Parameters:

objectiveObjectiveFunction

Objective function.

paramsParams

Parameter tensors.

step_dirParams

Vector v.

Returns:

torch.Tensor

Scalar value vᵀ H v.