系统工程与电子技术 ›› 2019, Vol. 41 ›› Issue (5): 1021-1027.doi: 10.3969/j.issn.1001-506X.2019.05.13

• 系统工程 • 上一篇    下一篇

基于Bagging-Down SGD算法的分布式深度网络

秦超1, 高晓光1, 陈大庆2   

  1. 1. 西北工业大学电子信息学院, 陕西 西安 710100; 2. 南岸大学, 英国 伦敦 SE10AA
  • 出版日期:2019-04-30 发布日期:2019-04-28

Distributed deep networks based on Bagging-Down SGD algorithm

QIN Chao1, GAO Xiaoguang1, CHEN Daqing2   

  1. 1. School of Electronics and Information, Northwestern Poly-technical University,
    Xi’an 710100, China; 2. London South Bank University, London SE10AA, England
  • Online:2019-04-30 Published:2019-04-28

PDF (PC)

摘要：

通过对大量数据进行训练并采用分布式深度学习算法可以学习到比较好的数据结构,而传统的分布式深度学习算法在处理大数据集时存在训练时间比较慢或者训练精度比较低的问题。提出Bootstrap向下聚合随机梯度下降(Bootstrap aggregating-down stochastic gradient descent,Bagging-Down SGD)算法重点来提高分布式深度网络的学习速率。Bagging-Down SGD算法通过在众多单机模型上加入速度控制器,对单机计算的参数值做统计处理,减少了参数更新的频率,并且可以使单机模型训练和参数更新在一定程度上分开,在保证训练精度的同时,提高了整个分布式模型的训练速度。该算法具有普适性,可以对多种类别的数据进行学习。

关键词: 深度网络, 分布式, Bootstrap向下聚合随机梯度下降, 速度控制器

Abstract:

As a cutting-edge disruptive technology, deep learning and unsupervised learning have attracted a significant research attention, and it has been widely acknowledged that training big data with a distributed deep learning algorithm can get better structures. However, there are two main problems with traditional distributed deep learning algorithms: the speed of training is slow and the accuracy of training is low. The Bootstrap aggregating-down stochastic gradient descent (Bagging-Down SGD) algorithm is proposed to solve the speed problem mainly. We add a speed controller to update the parameters of the single machine statistically, and to split model  training and parameters updating to improve the training speed with the assurance of the same accuracy. It is  to be proved in the experiment that the algorithm has the generality to learn the structures  of different kinds of data.

Key words: deep network, distributed, Bootstrap aggregating-down stochastic gradient descent (Bagging-Down SGD), speed controller

京ICP备11035014号-5
版权所有 © 2019 北京航天情报与信息研究所《系统工程与电子技术》编辑部
通讯地址：北京市142信箱32分箱《系统工程与电子技术》编辑部(100854)
电　话：(010)68388406/68386014    E-mail：xtgcydzjs@126.com(系统工程与电子技术)/jseeoffice@126.com(JSEE)
本系统由北京玛格泰克科技发展有限公司设计开发