torch_numopt.objective module

Defines the objective function interface and a supervised learning implementation.

The ObjectiveFunction abstract class provides the core abstraction for optimization: it wraps a loss function, handles parameter storage, and manages batched evaluation. The SupervisedLearningObjective specializes it for common machine learning tasks with data batching.

class ObjectiveFunction(params, optimizer, batched=False)

Bases: ABC

Abstract base class for an objective (loss) function.

An objective is callable and, when called, computes the loss and performs backpropagation to populate gradients of its parameters. This completely matches the closure used by pytorch optimizers, and this class can be passed to such optimizers as if it were a closure.

Attributes:

paramsParams

Tuple of parameter tensors to be optimized.

optimizertorch.optim.Optimizer

The optimizer that will use this objective (used for zeroing gradients).

batchedbool, optional

Whether the objective supports batching (i.e., the loss is evaluated over sub-sets of the data). Defaults to False.

Methods

__call__()	Call the objective, computing the loss and its gradients.
closure()	Compute the loss and backpropagate gradients.
loss(*params[, batch_idx])	Evaluate the objective function at the given parameters.
residual(*params[, batch_idx])	Compute the residual vector (e.g., prediction-target difference).

closure()

Compute the loss and backpropagate gradients.

Zeroes the gradients of the optimizer, evaluates the loss, and calls .backward() on it. This is the standard closure expected by many PyTorch optimizers.

Return type:

Tensor

Returns:

torch.Tensor

The scalar loss value with gradients attached.

abstract loss(*params, batch_idx=None)

Evaluate the objective function at the given parameters.

Return type:

Tensor

Parameters:

*paramsParams

Parameter tensors (must match the number and shape stored in self.params).

batch_idxint, optional

If the objective is batched, this index selects the batch. If None, the full dataset is used.

Returns:

torch.Tensor

Scalar loss value.

residual(*params, batch_idx=None)

Compute the residual vector (e.g., prediction-target difference).

This method is used by Gauss-Newton and similar algorithms that require the residual (not just the loss). The default implementation raises NotImplementedError.

Return type:

Tensor

Parameters:

*paramsParams

Parameter tensors.

batch_idxint, optional

Batch index for batched objectives.

Returns:

torch.Tensor

Residual tensor (shape depends on the problem).

class SupervisedLearningObjective(model, loss_fn, optimizer, weight_decay=0, batch_size=None)

Bases: ObjectiveFunction

Objective function for supervised learning problems.

This class wraps a PyTorch model, a loss function, and a data loader (X, y) to provide a standard objective. It supports mini-batch evaluation and L₂ weight decay.

Parameters:

modeltorch.nn.Module

The model whose parameters are to be optimized.

loss_fntorch.nn.Module

Loss function (e.g., torch.nn.MSELoss) that defines the criterion.

optimizertorch.optim.Optimizer

Optimizer used to zero gradients.

weight_decayfloat, default=0

Coefficient for L₂ regularization added to the loss.

batch_sizeint, optional

If provided, the objective will be evaluated in batches of this size (used for memory efficiency). If None, the whole dataset is used at once.

Methods

__call__()	Call the objective, computing the loss and its gradients.
batch_data_size(batch_idx)	Get the size of a specific batch (the last batch may be smaller).
closure()	Compute the loss and backpropagate gradients.
get_batch([batch_idx])	Retrieve the data slice corresponding to a batch.
loss(*params[, batch_idx])	Compute the supervised loss at the given parameters.
residual(*params[, batch_idx])	Compute the residual vector (model output minus target).
set_data(x, y)	Set the training data.

set_data(x, y)

Set the training data.

Parameters:

xtorch.Tensor

Input features.

ytorch.Tensor

Target labels.

batch_data_size(batch_idx)

Get the size of a specific batch (the last batch may be smaller).

Parameters:

batch_idxint

Batch index.

Returns:

int

Number of samples in that batch.

get_batch(batch_idx=None)

Retrieve the data slice corresponding to a batch.

Parameters:

batch_idxint, optional

Batch index. If None, the full dataset is returned.

Returns:

tuple (X, y)

Input and target tensors for the selected batch.

loss(*params, batch_idx=None)

Compute the supervised loss at the given parameters.

The loss is the sum (or mean, according to the loss function’s reduction) of the criterion over the selected batch, plus the weight-decay term.

Return type:

Tensor

Parameters:

*paramsParams

Parameter tensors.

batch_idxint, optional

Batch index; if None, use the full dataset.

Returns:

torch.Tensor

Scalar loss value.

residual(*params, batch_idx=None)

Compute the residual vector (model output minus target).

Return type:

Tensor

Parameters:

*paramsParams

Parameter tensors.

batch_idxint, optional

Batch index; if None, use the full dataset.

Returns:

torch.Tensor

Residual tensor of shape (batch_size, output_dim).