Initial Improvement of the Hybrid Accelerated Gradient Descent Process

Stefan Panic, Milena J. Petrovic, Miroslava Mihajlov Carevic

Research output: Contribution to journalArticle

Abstract

We improve the convergence properties of the iterative scheme for solving unconstrained optimisation problems introduced in Petrovic et al. ['Hybridization of accelerated gradient descent method', Numer. Algorithms (2017), doi:10.1007/s11075-017-0460-4] by optimising the value of the initial step length parameter in the backtracking line search procedure. We prove the validity of the algorithm and illustrate its advantages by numerical experiments and comparisons.

Original languageEnglish
Pages (from-to)331-338
Number of pages8
JournalBulletin of the Australian Mathematical Society
Volume98
Issue number2
DOIs
Publication statusPublished - 1 Oct 2018
Externally publishedYes

Fingerprint

Gradient Descent
Gradient Descent Method
Backtracking
Line Search
Unconstrained Optimization
Numerical Comparisons
Iterative Scheme
Convergence Properties
Numerical Experiment
Optimization Problem

Keywords

  • convergence rate
  • gradient descent methods
  • Line search
  • Newton method

ASJC Scopus subject areas

  • Mathematics(all)

Cite this

Initial Improvement of the Hybrid Accelerated Gradient Descent Process. / Panic, Stefan; Petrovic, Milena J.; Carevic, Miroslava Mihajlov.

In: Bulletin of the Australian Mathematical Society, Vol. 98, No. 2, 01.10.2018, p. 331-338.

Research output: Contribution to journalArticle

Panic, Stefan ; Petrovic, Milena J. ; Carevic, Miroslava Mihajlov. / Initial Improvement of the Hybrid Accelerated Gradient Descent Process. In: Bulletin of the Australian Mathematical Society. 2018 ; Vol. 98, No. 2. pp. 331-338.
@article{257ca7c3bc6b4384b3335b434f7a31bd,
title = "Initial Improvement of the Hybrid Accelerated Gradient Descent Process",
abstract = "We improve the convergence properties of the iterative scheme for solving unconstrained optimisation problems introduced in Petrovic et al. ['Hybridization of accelerated gradient descent method', Numer. Algorithms (2017), doi:10.1007/s11075-017-0460-4] by optimising the value of the initial step length parameter in the backtracking line search procedure. We prove the validity of the algorithm and illustrate its advantages by numerical experiments and comparisons.",
keywords = "convergence rate, gradient descent methods, Line search, Newton method",
author = "Stefan Panic and Petrovic, {Milena J.} and Carevic, {Miroslava Mihajlov}",
year = "2018",
month = "10",
day = "1",
doi = "10.1017/S0004972718000552",
language = "English",
volume = "98",
pages = "331--338",
journal = "Bulletin of the Australian Mathematical Society",
issn = "0004-9727",
publisher = "Cambridge University Press",
number = "2",

}

TY - JOUR

T1 - Initial Improvement of the Hybrid Accelerated Gradient Descent Process

AU - Panic, Stefan

AU - Petrovic, Milena J.

AU - Carevic, Miroslava Mihajlov

PY - 2018/10/1

Y1 - 2018/10/1

N2 - We improve the convergence properties of the iterative scheme for solving unconstrained optimisation problems introduced in Petrovic et al. ['Hybridization of accelerated gradient descent method', Numer. Algorithms (2017), doi:10.1007/s11075-017-0460-4] by optimising the value of the initial step length parameter in the backtracking line search procedure. We prove the validity of the algorithm and illustrate its advantages by numerical experiments and comparisons.

AB - We improve the convergence properties of the iterative scheme for solving unconstrained optimisation problems introduced in Petrovic et al. ['Hybridization of accelerated gradient descent method', Numer. Algorithms (2017), doi:10.1007/s11075-017-0460-4] by optimising the value of the initial step length parameter in the backtracking line search procedure. We prove the validity of the algorithm and illustrate its advantages by numerical experiments and comparisons.

KW - convergence rate

KW - gradient descent methods

KW - Line search

KW - Newton method

UR - http://www.scopus.com/inward/record.url?scp=85052695512&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85052695512&partnerID=8YFLogxK

U2 - 10.1017/S0004972718000552

DO - 10.1017/S0004972718000552

M3 - Article

VL - 98

SP - 331

EP - 338

JO - Bulletin of the Australian Mathematical Society

JF - Bulletin of the Australian Mathematical Society

SN - 0004-9727

IS - 2

ER -