A variable smoothing algorithm for solving convex optimization problems

In this article, we propose a method for solving unconstrained optimization problems with convex and Lipschitz continuous objective functions. By making use of the Moreau envelopes of the functions occurring in the objective, we smooth them to convex and differentiable functions with Lipschitz continuous gradients using both variable and constant smoothing parameters. The resulting problem is solved via an accelerated first-order method and this allows us to recover approximately the optimal solutions to the initial optimization problem with a rate of convergence of order <InlineEquation ID="IEq1"> <EquationSource Format="TEX">$$\mathcal {O}\left( \tfrac{\ln k}{k}\right) $$</EquationSource> <EquationSource Format="MATHML"> <math xmlns:xlink="http://www.w3.org/1999/xlink"> <mrow> <mi mathvariant="script">O</mi> <mfenced close=")" open="("> <mstyle displaystyle="false" scriptlevel="0"> <mfrac> <mrow> <mo>ln</mo> <mi>k</mi> </mrow> <mi>k</mi> </mfrac> </mstyle> </mfenced> </mrow> </math> </EquationSource> </InlineEquation> for variable smoothing and of order <InlineEquation ID="IEq2"> <EquationSource Format="TEX">$$\mathcal {O}\left( \tfrac{1}{k}\right) $$</EquationSource> <EquationSource Format="MATHML"> <math xmlns:xlink="http://www.w3.org/1999/xlink"> <mrow> <mi mathvariant="script">O</mi> <mfenced close=")" open="("> <mstyle displaystyle="false" scriptlevel="0"> <mfrac> <mn>1</mn> <mi>k</mi> </mfrac> </mstyle> </mfenced> </mrow> </math> </EquationSource> </InlineEquation> for constant smoothing. Some numerical experiments employing the variable smoothing method in image processing and in supervised learning classification are also presented. Copyright Sociedad de Estadística e Investigación Operativa 2015