Hybridization of accelerated gradient descent method

Milena Petrović, Vladimir Rakočević, Nataša Kontrec, Stefan Panić, Dejan Ilić

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

We present a gradient descent algorithm with a line search procedure for solving unconstrained optimization problems which is defined as a result of applying Picard-Mann hybrid iterative process on accelerated gradient descent SM method described in Stanimirović and Miladinović (Numer. Algor. 54, 503–520, 2010). Using merged features of both analyzed models, we show that new accelerated gradient descent model converges linearly and faster then the starting SM method which is confirmed trough displayed numerical test results. Three main properties are tested: number of iterations, CPU time and number of function evaluations. The efficiency of the proposed iteration is examined for the several values of the correction parameter introduced in Khan (2013).

Original languageEnglish
Pages (from-to)769-786
Number of pages18
JournalNumerical Algorithms
Volume79
Issue number3
DOIs
Publication statusPublished - 1 Nov 2018
Externally publishedYes

Fingerprint

Gradient Descent Method
Gradient Descent
Function evaluation
Program processors
Iteration
Descent Algorithm
Gradient Algorithm
Line Search
Unconstrained Optimization
Evaluation Function
Iterative Process
CPU Time
Linearly
Optimization Problem
Converge
Model

Keywords

  • Convergence rate
  • Gradient descent methods
  • Line search
  • Newton method

ASJC Scopus subject areas

  • Applied Mathematics

Cite this

Petrović, M., Rakočević, V., Kontrec, N., Panić, S., & Ilić, D. (2018). Hybridization of accelerated gradient descent method. Numerical Algorithms, 79(3), 769-786. https://doi.org/10.1007/s11075-017-0460-4

Hybridization of accelerated gradient descent method. / Petrović, Milena; Rakočević, Vladimir; Kontrec, Nataša; Panić, Stefan; Ilić, Dejan.

In: Numerical Algorithms, Vol. 79, No. 3, 01.11.2018, p. 769-786.

Research output: Contribution to journalArticle

Petrović, M, Rakočević, V, Kontrec, N, Panić, S & Ilić, D 2018, 'Hybridization of accelerated gradient descent method', Numerical Algorithms, vol. 79, no. 3, pp. 769-786. https://doi.org/10.1007/s11075-017-0460-4
Petrović, Milena ; Rakočević, Vladimir ; Kontrec, Nataša ; Panić, Stefan ; Ilić, Dejan. / Hybridization of accelerated gradient descent method. In: Numerical Algorithms. 2018 ; Vol. 79, No. 3. pp. 769-786.
@article{ae9388eb1ec44a0b884dd779b30b9028,
title = "Hybridization of accelerated gradient descent method",
abstract = "We present a gradient descent algorithm with a line search procedure for solving unconstrained optimization problems which is defined as a result of applying Picard-Mann hybrid iterative process on accelerated gradient descent SM method described in Stanimirović and Miladinović (Numer. Algor. 54, 503–520, 2010). Using merged features of both analyzed models, we show that new accelerated gradient descent model converges linearly and faster then the starting SM method which is confirmed trough displayed numerical test results. Three main properties are tested: number of iterations, CPU time and number of function evaluations. The efficiency of the proposed iteration is examined for the several values of the correction parameter introduced in Khan (2013).",
keywords = "Convergence rate, Gradient descent methods, Line search, Newton method",
author = "Milena Petrović and Vladimir Rakočević and Nataša Kontrec and Stefan Panić and Dejan Ilić",
year = "2018",
month = "11",
day = "1",
doi = "10.1007/s11075-017-0460-4",
language = "English",
volume = "79",
pages = "769--786",
journal = "Numerical Algorithms",
issn = "1017-1398",
publisher = "Springer Netherlands",
number = "3",

}

TY - JOUR

T1 - Hybridization of accelerated gradient descent method

AU - Petrović, Milena

AU - Rakočević, Vladimir

AU - Kontrec, Nataša

AU - Panić, Stefan

AU - Ilić, Dejan

PY - 2018/11/1

Y1 - 2018/11/1

N2 - We present a gradient descent algorithm with a line search procedure for solving unconstrained optimization problems which is defined as a result of applying Picard-Mann hybrid iterative process on accelerated gradient descent SM method described in Stanimirović and Miladinović (Numer. Algor. 54, 503–520, 2010). Using merged features of both analyzed models, we show that new accelerated gradient descent model converges linearly and faster then the starting SM method which is confirmed trough displayed numerical test results. Three main properties are tested: number of iterations, CPU time and number of function evaluations. The efficiency of the proposed iteration is examined for the several values of the correction parameter introduced in Khan (2013).

AB - We present a gradient descent algorithm with a line search procedure for solving unconstrained optimization problems which is defined as a result of applying Picard-Mann hybrid iterative process on accelerated gradient descent SM method described in Stanimirović and Miladinović (Numer. Algor. 54, 503–520, 2010). Using merged features of both analyzed models, we show that new accelerated gradient descent model converges linearly and faster then the starting SM method which is confirmed trough displayed numerical test results. Three main properties are tested: number of iterations, CPU time and number of function evaluations. The efficiency of the proposed iteration is examined for the several values of the correction parameter introduced in Khan (2013).

KW - Convergence rate

KW - Gradient descent methods

KW - Line search

KW - Newton method

UR - http://www.scopus.com/inward/record.url?scp=85039840490&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85039840490&partnerID=8YFLogxK

U2 - 10.1007/s11075-017-0460-4

DO - 10.1007/s11075-017-0460-4

M3 - Article

VL - 79

SP - 769

EP - 786

JO - Numerical Algorithms

JF - Numerical Algorithms

SN - 1017-1398

IS - 3

ER -