tick.solver.BFGS¶

class tick.solver.BFGS(tol: float = 1e-10, max_iter: int = 10, verbose: bool = True, print_every: int = 1, record_every: int = 1)[source]¶

Broyden, Fletcher, Goldfarb, and Shanno algorithm

This solver is actually a simple wrapping of scipy.optimize.fmin_bfgs BFGS (Broyden, Fletcher, Goldfarb, and Shanno) algorithm. This is a quasi-newton algotithm that builds iteratively approximations of the inverse Hessian. This solver can be used to minimize objectives of the form

\[f(w) + g(w),\]

for \(f\) with a smooth gradient and only \(g\) corresponding to the zero penalization (namely ProxZero) or ridge penalization (namely ProxL2sq). Function \(f\) corresponds to the model.loss method of the model (passed with set_model to the solver) and \(g\) corresponds to the prox.value method of the prox (passed with the set_prox method). The iterations stop whenever tolerance tol is achieved, or after max_iter iterations. The obtained solution \(w\) is returned by the solve method, and is also stored in the solution attribute of the solver.

Parameters:

tol : float, default=1e-10

The tolerance of the solver (iterations stop when the stopping criterion is below it)

max_iter : int, default=10

Maximum number of iterations of the solver

verbose : bool, default=True

If True, solver verboses history, otherwise nothing is displayed, but history is recorded anyway

print_every : int, default=10

Print history information every time the iteration number is a multiple of print_every. Used only is verbose is True

record_every : int, default=1

Save history information every time the iteration number is a multiple of record_every

Attributes:

model : Model

The model used by the solver, passed with the set_model method

prox : Prox

Proximal operator used by the solver, passed with the set_prox method

solution : numpy.array, shape=(n_coeffs,)

Minimizer found by the solver

history : dict-like

A dict-type of object that contains history of the solver along iterations. It should be accessed using the get_history method

time_start : str

Start date of the call to solve()

time_elapsed : float

Duration of the call to solve(), in seconds

time_end : str

End date of the call to solve()

dtype : {'float64', 'float32'}, default=’float64’

Type of the arrays used. This value is set from model and prox dtypes.

References

• Quasi-Newton method of Broyden, Fletcher, Goldfarb and Shanno (BFGS), see Wright, and Nocedal ‘Numerical Optimization’, 1999, pg. 198.

Examples using tick.solver.BFGS¶