torch_numopt.utils.param_operations module#
- param_zero_like(params)[source]#
Create a tuple of zero tensors with the same shapes as the input parameters.
- Parameters:
- paramsParams
Iterable of tensors providing the shapes.
- Returns:
- tuple of torch.Tensor
A tuple of tensors, each filled with zeros and with the same shape and device as the corresponding input tensor.
- param_add(params_a, params_b)[source]#
Element-wise addition of two parameter groups.
- Parameters:
- params_aParams
First iterable of tensors.
- params_bParams
Second iterable of tensors, must have the same structure as params_a.
- Returns:
- tuple of torch.Tensor
A new tuple where each element is the sum of the corresponding elements of params_a and params_b.
- param_scalar_prod(scalar, params)[source]#
Multiplies every tensor in a parameter group by a scalar.
- Parameters:
- scalarfloat
The scalar multiplier.
- paramsParams
Iterable of tensors to scale.
- Returns:
- tuple of torch.Tensor
New tuple containing each tensor multiplied by scalar.
- param_dot(params_a, params_b)[source]#
Computes the dot product between two parameter groups treated as a single flat vector.
- Parameters:
- params_aParams
First iterable of tensors.
- params_bParams
Second iterable of tensors, same shapes as params_a.
- Returns:
- torch.Tensor
Scalar tensor equal to sum_{p_a, p_b} (p_a * p_b).sum().
- param_diff(params_a, params_b)[source]#
Element-wise subtraction of two parameter groups.
- Parameters:
- params_aParams
First iterable of tensors.
- params_bParams
Second iterable of tensors, same structure.
- Returns:
- tuple of torch.Tensor
New tuple where each element is params_a[i] - params_b[i].
- param_scaled_add(params_a, params_b, scale)[source]#
Computes params_a + scale * params_b element-wise.
- Parameters:
- params_aParams
Base parameter group.
- params_bParams
Parameter group to be scaled and added.
- scalefloat
Scaling factor for params_b.
- Returns:
- tuple of torch.Tensor
New tuple with the result of the scaled addition.
- param_norm(params)[source]#
Euclidean (L2) norm of a parameter group treated as a flat vector.
- Parameters:
- paramsParams
Iterable of tensors.
- Returns:
- torch.Tensor
Scalar tensor equal to sqrt(sum_{p} (p * p).sum()).
- param_mult(params_a, params_b)[source]#
Element-wise (Hadamard) product of two parameter groups.
- Parameters:
- params_aParams
First iterable of tensors.
- params_bParams
Second iterable of tensors, same shapes.
- Returns:
- tuple of torch.Tensor
New tuple where each element is params_a[i] * params_b[i].
- param_transpose(params)[source]#
Transposes every 2-dimensional tensor in the parameter group.
- Parameters:
- paramsParams
Iterable of tensors; only 2D tensors are transposed, others are left unchanged.
- Returns:
- tuple of torch.Tensor
New tuple with transposed tensors where applicable.
- param_neg(params)[source]#
Negates every tensor in a parameter group.
- Parameters:
- paramsParams
Iterable of tensors.
- Returns:
- tuple of torch.Tensor
New tuple with each element negated (-p).
- param_copy(params)[source]#
Creates a deep copy of a parameter group (each tensor cloned).
- Parameters:
- paramsParams
Iterable of tensors to copy.
- Returns:
- tuple of torch.Tensor
New tuple containing detached clones of the original tensors.
- param_is_finite(params)[source]#
Checks that all elements in a parameter group are finite.
- Parameters:
- paramsParams
Iterable of tensors to inspect.
- Returns:
- bool
True if every element of every tensor is finite, False otherwise.
- param_sizes(params)[source]#
Obtains the shape of every matrix in the parameters provided.
- Parameters:
- params: Params
Sequence of matrices containing a sequence of parameters.
- param_reshape_like(params_flat, params)[source]#
Reshapes a vector into a sequence of matrices with the same shapes as the params parameter.
- Parameters:
- params_flat: Tensor
Vector with the parameters to reshape.
- params: Params
Sequence of matrices with the desired shape.
- Returns:
- reshaped_params: Tensor
- param_flatten(params)[source]#
Flattens an entire parameter group into a single 1-dimensional tensor.
- Parameters:
- paramsParams
Iterable of tensors (possibly nested). Each tensor is flattened and concatenated.
- Returns:
- torch.Tensor
1-D tensor containing all parameter values concatenated in order.
- param_numel(params)[source]#
Returns the total amount of parameters.
- Return type:
int- Parameters:
- paramsParams
Iterable of tensors.
- Returns:
- int
Count of the total number of parameters.