Function Wrappers

Flattening and folding the input and outputs of a function

class paragami.function_patterns.TransformFunctionInput(original_fun, patterns, free, original_is_flat, argnums=None)

Convert a function of folded (or flattened) values into one that takes flattened (or folded) values.

Examples

mat_pattern = paragami.PSDSymmetricMatrixPattern(3)

def fun(offset, mat, kwoffset=3):
    return np.linalg.slogdet(mat + offset + kwoffset)[1]

flattened_fun = paragami.TransformFunctionInput(
    original_fun=fun, patterns=mat_pattern,
    free=True, argnums=1, original_is_flat=False)

# pd_mat is a matrix:
pd_mat = np.eye(3) + np.full((3, 3), 0.1)

# pd_mat_flat is an unconstrained vector:
pd_mat_flat = mat_pattern.flatten(pd_mat, free=True)

# These two functions return the same value:
print('Original: {}'.format(
      fun(2, pd_mat, kwoffset=3)))
print('Flat: {}'.format(
      flattened_fun(2, pd_mat_flat, kwoffset=3)))

__init__(original_fun, patterns, free, original_is_flat, argnums=None)

Parameters:

original_fun: callable

A function that takes one or more values as input.

patterns: `paragami.Pattern` or list of `paragami.PatternPattern`

A single pattern or array of patterns describing the input to original_fun.

free: `bool` or list of `bool`

Whether or not the corresponding elements of patterns should use free or non-free flattened values.

original_is_flat: `bool`

If True, convert original_fun from taking flat arguments to one taking folded arguments. If False, convert original_fun from taking folded arguments to one taking flat arguments.

argnums: `int` or list of `int`

The 0-indexed locations of the corresponding pattern in patterns in the order of the arguments fo original_fun.

class paragami.function_patterns.FlattenFunctionInput(original_fun, patterns, free, argnums=None)

A convenience wrapper of paragami.TransformFunctionInput.

See also

paragami.TransformFunctionInput

__init__(original_fun, patterns, free, argnums=None)

Parameters:

original_fun: callable

A function that takes one or more values as input.

patterns: `paragami.Pattern` or list of `paragami.PatternPattern`

A single pattern or array of patterns describing the input to original_fun.

free: `bool` or list of `bool`

Whether or not the corresponding elements of patterns should use free or non-free flattened values.

original_is_flat: `bool`

If True, convert original_fun from taking flat arguments to one taking folded arguments. If False, convert original_fun from taking folded arguments to one taking flat arguments.

argnums: `int` or list of `int`

The 0-indexed locations of the corresponding pattern in patterns in the order of the arguments fo original_fun.

class paragami.function_patterns.FoldFunctionInput(original_fun, patterns, free, argnums=None)

A convenience wrapper of paragami.TransformFunctionInput.

See also

paragami.TransformFunctionInput

__init__(original_fun, patterns, free, argnums=None)

Parameters:

original_fun: callable

A function that takes one or more values as input.

patterns: `paragami.Pattern` or list of `paragami.PatternPattern`

A single pattern or array of patterns describing the input to original_fun.

free: `bool` or list of `bool`

Whether or not the corresponding elements of patterns should use free or non-free flattened values.

original_is_flat: `bool`

If True, convert original_fun from taking flat arguments to one taking folded arguments. If False, convert original_fun from taking folded arguments to one taking flat arguments.

argnums: `int` or list of `int`

The 0-indexed locations of the corresponding pattern in patterns in the order of the arguments fo original_fun.

class paragami.function_patterns.FlattenFunctionOutput(original_fun, patterns, free, retnums=None)

A convenience wrapper of paragami.TransformFunctionOutput.

See also

paragami.TransformFunctionOutput

__init__(original_fun, patterns, free, retnums=None)

Parameters:

original_fun: callable

A function that returns one or more values.

patterns: `paragami.Pattern` or list of `paragami.PatternPattern`

A single pattern or array of patterns describing the return value of original_fun.

free: `bool` or list of `bool`

Whether or not the corresponding elements of patterns should use free or non-free flattened values.

original_is_flat: `bool`

If True, convert original_fun from returning flat values to one returning folded values. If False, convert original_fun from returning folded values to one returning flat values.

retnums: `int` or list of `int`

The 0-indexed locations of the corresponding pattern in patterns in the order of the return values of original_fun.

class paragami.function_patterns.FoldFunctionOutput(original_fun, patterns, free, retnums=None)

A convenience wrapper of paragami.TransformFunctionOutput.

See also

paragami.TransformFunctionOutput

__init__(original_fun, patterns, free, retnums=None)

Parameters:

original_fun: callable

A function that returns one or more values.

patterns: `paragami.Pattern` or list of `paragami.PatternPattern`

A single pattern or array of patterns describing the return value of original_fun.

free: `bool` or list of `bool`

Whether or not the corresponding elements of patterns should use free or non-free flattened values.

original_is_flat: `bool`

If True, convert original_fun from returning flat values to one returning folded values. If False, convert original_fun from returning folded values to one returning flat values.

retnums: `int` or list of `int`

The 0-indexed locations of the corresponding pattern in patterns in the order of the return values of original_fun.

class paragami.function_patterns.FoldFunctionInputAndOutput(original_fun, input_patterns, input_free, input_argnums, output_patterns, output_free, output_retnums=None)

A convenience wrapper of paragami.FoldFunctionInput and paragami.FoldFunctionOutput.

See also

paragami.FoldFunctionInput, paragami.FoldFunctionOutput

__init__(original_fun, input_patterns, input_free, input_argnums, output_patterns, output_free, output_retnums=None)

Initialize self. See help(type(self)) for accurate signature.

class paragami.function_patterns.FlattenFunctionInputAndOutput(original_fun, input_patterns, input_free, input_argnums, output_patterns, output_free, output_retnums=None)

A convenience wrapper of paragami.FlattenFunctionInput and paragami.FlattenFunctionOutput.

See also

paragami.FlattenFunctionInput, paragami.FlattenFunctionOutput

__init__(original_fun, input_patterns, input_free, input_argnums, output_patterns, output_free, output_retnums=None)

Initialize self. See help(type(self)) for accurate signature.

Preconditioning an objective function

class paragami.optimization_lib.PreconditionedFunction(original_fun)

Get a function whose input has been preconditioned.

Throughout, the subscript _c will denote quantiites or funcitons in the preconditioned space. For example, x will refer to a variable in the original space and x_c to the same variable after preconditioning.

Preconditioning means transforming \(x \rightarrow x_c = A^{-1} x\), where the matrix \(A\) is the “preconditioner”. If \(f\) operates on \(x\), then the preconditioned function operates on \(x_c\) and is defined by \(f_c(x_c) := f(A x_c) = f(x)\). Gradients of the preconditioned function are defined with respect to its argument in the preconditioned space, e.g., \(f'_c = \frac{df_c}{dx_c}\).

A typical value of the preconditioner is an inverse square root of the Hessian of \(f\), because then the Hessian of \(f_c\) is the identity when the gradient is zero. This can help speed up the convergence of optimization algorithms.

Methods

set_preconditioner:	Set the preconditioner to a specified value.
set_preconditioner_with_hessian:	Set the preconditioner based on the Hessian of the objective at a point in the orginal domain.
precondition:	Convert from the original domain to the preconditioned domain.
unprecondition:	Convert from the preconditioned domain to the original domain.

__init__(original_fun)

Parameters:

original_fun:

callable function of a single argument

preconditioner:

The initial preconditioner.

preconditioner_inv:

The inverse of the initial preconditioner.

get_preconditioner(dim)

Return a matrix representation of the preconditioner. This may be expensive.

get_preconditioner_inv(dim)

Return a matrix representation of the preconditioner. This may be expensive.

precondition(x)

Multiply by the inverse of the preconditioner to convert \(x\) in the original domain to \(x_c\) in the preconditioned domain.

This function is provided for convenience, but it is more numerically stable to use np.linalg.solve(preconditioner, x).

set_preconditioner_functions(a_times, a_inv_times)

Set the preconditioner with a functions that perform matrix multiplication.

set_preconditioner_matrix(a, a_inv=None)

Set the preconditioner with a matrix.

set_preconditioner_with_hessian(x=None, hessian=None, ev_min=None, ev_max=None)

Set the precoditioner to the inverse square root of the Hessian of the original objective (or an approximation thereof).

Parameters:

x: Numeric vector

The point at which to evaluate the Hessian of original_fun. If x is specified, the Hessian is evaluated with automatic differentiation. Specify either x or hessian but not both.

hessian: Numeric matrix

The hessian of original_fun or an approximation of it. Specify either x or hessian but not both.

ev_min: float

If not None, set eigenvaluse of hessian that are less than ev_min to ev_min before taking the square root.

ev_maxs: float

If not None, set eigenvaluse of hessian that are greater than ev_max to ev_max before taking the square root.

Returns:

Sets the precoditioner for the class and returns the Hessian with

the eigenvalues thresholded by ``ev_min`` and ``ev_max``.

unprecondition(x_c)

Multiply by the preconditioner to convert \(x_c\) in the preconditioned domain to \(x\) in the original domain.

An optimization objective class

class paragami.optimization_lib.OptimizationObjective(objective_fun, print_every=1, log_every=0)

Derivatives and logging for an optimization objective function.

Attributes:

optimization_log: Dictionary

A record of the optimization progress as recorded by log_value.

Methods

f:	The objective function with logging.
grad:	The gradient of the objective function.
hessian:	The Hessian of the objective function.
hessian_vector_product:	The Hessian vector product of the objective function.
set_print_every:	Set how often to display optimization progress.
set_log_every:	Set how often to log optimization progress.
reset_iteration_count:	Reset the number of iterations for the purpose of printing and logging.
reset_log:	Clear the log.
reset:	Run reset_iteration_count and reset_log.
print_value:	Display a function evaluation.
log_value:	Log a function evaluation.

__init__(objective_fun, print_every=1, log_every=0)

Parameters:

obj_fun: Callable function of one argumnet

The function to be minimized.

print_every: integer

Print the optimization value every print_every iterations.

log_every: integer

Log the optimization value every log_every iterations.

log_value(num_f_evals, x, f_val)

Log the optimization progress. To create a custom log, overload this function. By default, the log is a list of tuples (iteration, x, f(x)).

Parameters:

num_f_vals: Integer

The total number of function evaluations.

x:

The current argument to the objective function.

f_val:

The value of the objective at x.

num_iterations()

Return the number of times the optimization function has been called, not counting any derivative evaluations.

print_value(num_f_evals, x, f_val)

Display the optimization progress. To display a custom update, overload this function.

Parameters:

num_f_vals: Integer

The total number of function evaluations.

x:

The current argument to the objective function.

f_val:

The value of the objective at x.

reset()

Reset the itreation count and clear the log.

set_log_every(n)

Parameters:

n: integer

Log the objective function value every n iterations. If 0, do not log.

set_print_every(n)

Parameters:

n: integer

Print the objective function value every n iterations. If 0, do not print any output.