tick.solver.
BFGS
(tol: float = 1e-10, max_iter: int = 10, verbose: bool = True, print_every: int = 1, record_every: int = 1)[source]¶Broyden, Fletcher, Goldfarb, and Shanno algorithm
This solver is actually a simple wrapping of scipy.optimize.fmin_bfgs
BFGS (Broyden, Fletcher, Goldfarb, and Shanno) algorithm. This is a
quasi-newton algotithm that builds iteratively approximations of the inverse
Hessian. This solver can be used to minimize objectives of the form
for \(f\) with a smooth gradient and only \(g\) corresponding to
the zero penalization (namely ProxZero
)
or ridge penalization (namely ProxL2sq
).
Function \(f\) corresponds to the model.loss
method of the model
(passed with set_model
to the solver) and \(g\) corresponds to
the prox.value
method of the prox (passed with the set_prox
method).
The iterations stop whenever tolerance tol
is achieved, or
after max_iter
iterations. The obtained solution \(w\) is returned
by the solve
method, and is also stored in the solution
attribute
of the solver.
tol : float
, default=1e-10
The tolerance of the solver (iterations stop when the stopping criterion is below it)
max_iter : int
, default=10
Maximum number of iterations of the solver
verbose : bool
, default=True
If
True
, solver verboses history, otherwise nothing is displayed, but history is recorded anyway
print_every : int
, default=10
Print history information every time the iteration number is a multiple of
print_every
. Used only isverbose
is True
record_every : int
, default=1
Save history information every time the iteration number is a multiple of
record_every
model : Model
The model used by the solver, passed with the
set_model
method
prox : Prox
Proximal operator used by the solver, passed with the
set_prox
method
solution : numpy.array
, shape=(n_coeffs,)
Minimizer found by the solver
history : dict
-like
A dict-type of object that contains history of the solver along iterations. It should be accessed using the
get_history
method
time_start : str
Start date of the call to
solve()
time_elapsed : float
Duration of the call to
solve()
, in seconds
time_end : str
End date of the call to
solve()
dtype : {'float64', 'float32'}
, default=’float64’
Type of the arrays used. This value is set from model and prox dtypes.
References
Quasi-Newton method of Broyden, Fletcher, Goldfarb and Shanno (BFGS), see Wright, and Nocedal ‘Numerical Optimization’, 1999, pg. 198.
__init__
(tol: float = 1e-10, max_iter: int = 10, verbose: bool = True, print_every: int = 1, record_every: int = 1)[source]¶Initialize self. See help(type(self)) for accurate signature.
get_history
(key=None)¶Returns history of the solver
key : str
, default=None
If
None
all history is returned as adict
If
str
, name of the history element to retrieve
output : list
or dict
If
key
is None orkey
is not in history then output is a dict containing history of all keysIf
key
is the name of an element in the history, output is alist
containing the history of this element
objective
(coeffs, loss: float = None)¶Compute the objective function
coeffs : np.array
, shape=(n_coeffs,)
Point where the objective is computed
loss : float
, default=`None`
Gives the value of the loss if already known (allows to avoid its computation in some cases)
output : float
Value of the objective at given
coeffs
set_model
(model: tick.base_model.model.Model)[source]¶Set model in the solver
model : Model
Sets the model in the solver. The model gives the first order information about the model (loss, gradient, among other things)
output : Solver
The
Solver
with given model
set_prox
(prox: tick.prox.base.prox.Prox)[source]¶Set proximal operator in the solver.
prox : Prox
The proximal operator of the penalization function
output : Solver
The solver with given prox
Notes
In some solvers, set_model
must be called before
set_prox
, otherwise and error might be raised.
tick.solver.BFGS
¶