An online conjugate gradient algorithm for large-scale data analysis in machine learning
In recent years, the amount of available data is growing exponentially, and large-scale data is becoming ubiquitous. Machine learning is a key to deriving insight from this deluge of data. In this paper, we focus on the large-scale data analysis, especially classification data, and propose an online...
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
AIMS Press
2021-12-01
|
Series: | AIMS Mathematics |
Subjects: | |
Online Access: | http://awstest.aimspress.com/article/doi/10.3934/math.2021092?viewType=HTML |
id |
doaj-fc37ba4dd22344e4a6850136e0d4c8ba |
---|---|
record_format |
Article |
spelling |
doaj-fc37ba4dd22344e4a6850136e0d4c8ba2020-12-15T01:09:11ZengAIMS PressAIMS Mathematics2473-69882021-12-01621515153710.3934/math.2021092An online conjugate gradient algorithm for large-scale data analysis in machine learningWei Xue0Pengcheng Wan1Qiao Li 2Ping Zhong3Gaohang Yu4Tao Tao5 1. School of Computer Science and Technology, Anhui University of Technology, Maanshan 243032, China 2. National Key Laboratory of Science and Technology on Automatic Target Recognition, National University of Defense Technology, Changsha 410073, China 3. Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 210096, China1. School of Computer Science and Technology, Anhui University of Technology, Maanshan 243032, China1. School of Computer Science and Technology, Anhui University of Technology, Maanshan 243032, China2. National Key Laboratory of Science and Technology on Automatic Target Recognition, National University of Defense Technology, Changsha 410073, China4. School of Sciences, Hangzhou Dianzi University, Hangzhou 310018, China1. School of Computer Science and Technology, Anhui University of Technology, Maanshan 243032, ChinaIn recent years, the amount of available data is growing exponentially, and large-scale data is becoming ubiquitous. Machine learning is a key to deriving insight from this deluge of data. In this paper, we focus on the large-scale data analysis, especially classification data, and propose an online conjugate gradient (CG) descent algorithm. Our algorithm draws from a recent improved Fletcher-Reeves (IFR) CG method proposed in Jiang and Jian[13] as well as a recent approach to reduce variance for stochastic gradient descent from Johnson and Zhang [15]. In theory, we prove that the proposed online algorithm achieves a linear convergence rate under strong Wolfe line search when the objective function is smooth and strongly convex. Comparison results on several benchmark classification datasets demonstrate that our approach is promising in solving large-scale machine learning problems, viewed from the points of area under curve (AUC) value and convergence behavior.http://awstest.aimspress.com/article/doi/10.3934/math.2021092?viewType=HTMLmachine learningonline learningstochastic optimizationconjugate gradientvariance reduction |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Wei Xue Pengcheng Wan Qiao Li Ping Zhong Gaohang Yu Tao Tao |
spellingShingle |
Wei Xue Pengcheng Wan Qiao Li Ping Zhong Gaohang Yu Tao Tao An online conjugate gradient algorithm for large-scale data analysis in machine learning AIMS Mathematics machine learning online learning stochastic optimization conjugate gradient variance reduction |
author_facet |
Wei Xue Pengcheng Wan Qiao Li Ping Zhong Gaohang Yu Tao Tao |
author_sort |
Wei Xue |
title |
An online conjugate gradient algorithm for large-scale data analysis in machine learning |
title_short |
An online conjugate gradient algorithm for large-scale data analysis in machine learning |
title_full |
An online conjugate gradient algorithm for large-scale data analysis in machine learning |
title_fullStr |
An online conjugate gradient algorithm for large-scale data analysis in machine learning |
title_full_unstemmed |
An online conjugate gradient algorithm for large-scale data analysis in machine learning |
title_sort |
online conjugate gradient algorithm for large-scale data analysis in machine learning |
publisher |
AIMS Press |
series |
AIMS Mathematics |
issn |
2473-6988 |
publishDate |
2021-12-01 |
description |
In recent years, the amount of available data is growing exponentially, and large-scale data is becoming ubiquitous. Machine learning is a key to deriving insight from this deluge of data. In this paper, we focus on the large-scale data analysis, especially classification data, and propose an online conjugate gradient (CG) descent algorithm. Our algorithm draws from a recent improved Fletcher-Reeves (IFR) CG method proposed in Jiang and Jian[13] as well as a recent approach to reduce variance for stochastic gradient descent from Johnson and Zhang [15]. In theory, we prove that the proposed online algorithm achieves a linear convergence rate under strong Wolfe line search when the objective function is smooth and strongly convex. Comparison results on several benchmark classification datasets demonstrate that our approach is promising in solving large-scale machine learning problems, viewed from the points of area under curve (AUC) value and convergence behavior. |
topic |
machine learning online learning stochastic optimization conjugate gradient variance reduction |
url |
http://awstest.aimspress.com/article/doi/10.3934/math.2021092?viewType=HTML |
work_keys_str_mv |
AT weixue anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT pengchengwan anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT qiaoli anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT pingzhong anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT gaohangyu anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT taotao anonlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT weixue onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT pengchengwan onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT qiaoli onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT pingzhong onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT gaohangyu onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning AT taotao onlineconjugategradientalgorithmforlargescaledataanalysisinmachinelearning |
_version_ |
1724382976968491008 |