2020arXiv (Cornell University)Open access

Fast adaptive by constants of strong-convexity and Lipschitz for\n gradient first order methods

Nikita O. Pletnev

Open full text 0 citations

Abstract

The work is devoted to the construction of efficient and applicable to real\ntasks first-order methods of convex optimization, that is, using only values of\nthe target function and its derivatives. Construction uses OGM-G, fast gradient\nmethod which is optimal by complexity, but requires to know the Lipschitz\nconstant for gradient and the strong convexity constant to determine the number\nof steps and step length. This requirement makes practical usage impossible. An\nadaptive on the constant for strong convexity algorithm ACGM is proposed, based\non restarts of the OGM-G with update of the strong convexity constant estimate,\nand an adaptive on the Lipschitz constant for gradient ALGM, in which the use\nof OGM-G restarts is supplemented by the selection of the Lipschitz constant\nwith verification of the convexity conditions used in the universal gradient\ndescent method. This eliminates the disadvantages of the original method\nassociated with the need to know these constants, which makes practical usage\npossible. Optimality of estimates for the complexity of the constructed\nalgorithms is proved. To verify the results obtained, experiments on model\nfunctions and real tasks from machine learning are carried out.\n

Open-access reader

About this research paper

What this paper is about

The work is devoted to the construction of efficient and applicable to real\ntasks first-order methods of convex optimization, that is, using only values of\nthe target function and its derivatives. Construction uses OGM-G, fast gradient\nmethod which is optimal by complexity, but requires to know the Lipschitz\nconstant for gradient and the strong convexity constant to determine the number\nof steps and step length. This requirement makes practical usage impossible. An\nadaptive on the constant for strong convexity algorithm ACGM is proposed, based\non restarts of the OGM-G with update of the strong convexity constant estimate,\nand an adaptive on the Lipschitz constant for gradient ALGM, in which the use\nof OGM-G restarts is supplemented by the selection of the Lipschitz constant\nwith verification of the convexity conditions used in the universal gradient\ndescent method. This eliminates the disadvantages of the original method\nassociated with the need to know these constants, which makes practical usage\npossible. Optimality of estimates for the complexity of the constructed\nalgorithms is proved. To verify the results obtained, experiments on model\nfunctions and real tasks from machine learning are carried out.\n

Why it matters

A significance statement is not available in the OpenAlex record.

Key contribution

A contribution statement is not available in the OpenAlex record.

Method / approach

Method details are not available in the OpenAlex metadata.

Main findings

Findings are not separately available in the OpenAlex metadata.

Limitations

Limitations are not available in the OpenAlex metadata.

Applications

Application details are not available in the OpenAlex metadata.

Available abstract

The work is devoted to the construction of efficient and applicable to real\ntasks first-order methods of convex optimization, that is, using only values of\nthe target function and its derivatives. Construction uses OGM-G, fast gradient\nmethod which is optimal by complexity, but requires to know the Lipschitz\nconstant for gradient and the strong convexity constant to determine the number\nof steps and step length. This requirement makes practical usage impossible. An\nadaptive on the constant for strong convexity algorithm ACGM is proposed, based\non restarts of the OGM-G with update of the strong convexity constant estimate,\nand an adaptive on the Lipschitz constant for gradient ALGM, in which the use\nof OGM-G restarts is supplemented by the selection of the Lipschitz constant\nwith verification of the convexity conditions used in the universal gradient\ndescent method. This eliminates the disadvantages of the original method\nassociated with the need to know these constants, which makes practical usage\npossible. Optimality of estimates for the complexity of the constructed\nalgorithms is proved. To verify the results obtained, experiments on model\nfunctions and real tasks from machine learning are carried out.\n

Key concepts: Lipschitz continuity, Convexity, Constant (computer programming), Gradient descent, Function (biology), Mathematical optimization, Convex function, Computer science

Related papers

Back to paper searchBrowse research topicsOriginal source
Fast adaptive by constants of strong-convexity and Lipschitz for\n gradient first order methods — Research Paper | ScholarLens