Hybridization of accelerated gradient descent method

Milena Petrović, Vladimir Rakočević, Nataša Kontrec, Stefan Panić, Dejan Ilić

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

We present a gradient descent algorithm with a line search procedure for solving unconstrained optimization problems which is defined as a result of applying Picard-Mann hybrid iterative process on accelerated gradient descent SM method described in Stanimirović and Miladinović (Numer. Algor. 54, 503–520, 2010). Using merged features of both analyzed models, we show that new accelerated gradient descent model converges linearly and faster then the starting SM method which is confirmed trough displayed numerical test results. Three main properties are tested: number of iterations, CPU time and number of function evaluations. The efficiency of the proposed iteration is examined for the several values of the correction parameter introduced in Khan (2013).

Original languageEnglish
Pages (from-to)769-786
Number of pages18
JournalNumerical Algorithms
Volume79
Issue number3
DOIs
Publication statusPublished - 1 Nov 2018
Externally publishedYes

Keywords

  • Convergence rate
  • Gradient descent methods
  • Line search
  • Newton method

ASJC Scopus subject areas

  • Applied Mathematics

Fingerprint Dive into the research topics of 'Hybridization of accelerated gradient descent method'. Together they form a unique fingerprint.

  • Cite this

    Petrović, M., Rakočević, V., Kontrec, N., Panić, S., & Ilić, D. (2018). Hybridization of accelerated gradient descent method. Numerical Algorithms, 79(3), 769-786. https://doi.org/10.1007/s11075-017-0460-4