tick.solver.AGD¶

class tick.solver.AGD(step: float = None, tol: float = 1e-10, max_iter: int = 100, linesearch: bool = True, linesearch_step_increase: float = 2.0, linesearch_step_decrease: float = 0.5, verbose: bool = True, print_every: int = 10, record_every: int = 1)[source]¶

Accelerated proximal gradient descent

For the minimization of objectives of the form

\[f(w) + g(w),\]

where \(f\) has a smooth gradient and \(g\) is prox-capable. Function \(f\) corresponds to the model.loss method of the model (passed with set_model to the solver) and \(g\) corresponds to the prox.value method of the prox (passed with the set_prox method). One iteration of AGD is as follows:

\[\begin{split}w^{k} &\gets \mathrm{prox}_{\eta g} \big(z^k - \eta \nabla f(z^k) \big) \\ t_{k+1} &\gets \frac{1 + \sqrt{1 + 4 t_k^2}}{2} \\ z^{k+1} &\gets w^k + \frac{t_k - 1}{t_{k+1}} (w^k - w^{k-1})\end{split}\]

where \(\nabla f(w)\) is the gradient of \(f\) given by the model.grad method and \(\mathrm{prox}_{\eta g}\) is given by the prox.call method. The step-size \(\eta\) can be tuned with step. The iterations stop whenever tolerance tol is achieved, or after max_iter iterations. The obtained solution \(w\) is returned by the solve method, and is also stored in the solution attribute of the solver.

Parameters:

step : float, default=None

Step-size parameter, the most important parameter of the solver. Whenever possible, this can be automatically tuned as step = 1 / model.get_lip_best(). If linesearch=True, this is the first step-size to be used in the linesearch (that should be taken as too large).

tol : float, default=1e-10

The tolerance of the solver (iterations stop when the stopping criterion is below it)

max_iter : int, default=100

Maximum number of iterations of the solver.

linesearch : bool, default=True

If True, use backtracking linesearch to tune the step automatically.

verbose : bool, default=True

If True, solver verboses history, otherwise nothing is displayed, but history is recorded anyway

print_every : int, default=10

Print history information every time the iteration number is a multiple of print_every. Used only is verbose is True

record_every : int, default=1

Save history information every time the iteration number is a multiple of record_every

linesearch_step_increase : float, default=2.

Factor of step increase when using linesearch

linesearch_step_decrease : float, default=0.5

Factor of step decrease when using linesearch

Attributes:

model : Model

The model used by the solver, passed with the set_model method

prox : Prox

Proximal operator used by the solver, passed with the set_prox method

solution : numpy.array, shape=(n_coeffs,)

Minimizer found by the solver

history : dict-like

A dict-type of object that contains history of the solver along iterations. It should be accessed using the get_history method

time_start : str

Start date of the call to solve()

time_elapsed : float

Duration of the call to solve(), in seconds

time_end : str

End date of the call to solve()

References

• A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM journal on imaging sciences, 2009

Examples using tick.solver.AGD¶