Science Explorer Back to mapMap
Discussion
Topic · Stochastic Gradient Optimization Techniques

Most cited papers in Stochastic Gradient Optimization Techniques

Stochastic Gradient Optimization Techniques

What are the most cited papers in Stochastic Gradient Optimization Techniques?

The most cited paper in Stochastic Gradient Optimization Techniques is "Adam: A Method for Stochastic Optimization" (Diederik P. Kingma, Jimmy Lei Ba, 2014), with 84,625 citations. Next come "Adam: A Method for Stochastic Optimization" (50,327) and "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization" (8,604).

Showing 50 of 100 papers. Every paper with more than 691 citations is listed.

Most cited papers, sortable by citations and release date
#Paper
1 Adam: A Method for Stochastic Optimization Diederik P. Kingma, Jimmy Lei Ba · arXiv (Cornell University) PreprintOpen access 22 Dec 2014 84,625
2 Adam: A Method for Stochastic Optimization Mandi, Jayanta, Canoy, Rocsildes, Bucarey, Víctor et al. · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) Open access May 2015 50,327
3 Adaptive Subgradient Methods for Online Learning and Stochastic Optimization John C. Duchi, Elad Hazan, Yoram Singer 2010 8,604
4 Deep Learning with Differential Privacy Martı́n Abadi, Andy Chu, Ian Goodfellow et al. Conference paperOpen access 24 Oct 2016 6,312
5 Large-Scale Machine Learning with Stochastic Gradient Descent Léon Bottou Book chapter 2010 5,757
6 Communication-Efficient Learning of Deep Networks from Decentralized Data H. Brendan McMahan, Eider Moore, Daniel Ramage et al. · arXiv (Cornell University) PreprintOpen access 17 Feb 2016 5,182
7 Greedy Layer-Wise Training of Deep Networks Yoshua Bengio, Pascal Lamblin, Dan Popovici et al. · The MIT Press eBooks Book chapter 7 Sep 2007 4,753
8 `` Direct Search'' Solution of Numerical and Statistical Problems Robert Hooke, T. A. Jeeves · Journal of the ACM Open access Apr 1961 4,285
9 Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions Nathan Halko, Per‐Gunnar Martinsson, Joel A. Tropp · SIAM Review 2011 4,178
10 Practical Secure Aggregation for Privacy-Preserving Machine Learning Keith Bonawitz, Vladimir Ivanov, Ben Kreuter et al. Conference paper 27 Oct 2017 3,808
11 Distributed Subgradient Methods for Multi-Agent Optimization Angelia Nedić, Asuman Ozdaglar · IEEE Transactions on Automatic Control 2009 3,800
12 On the importance of initialization and momentum in deep learning Ilya Sutskever, James Martens, George E. Dahl et al. 16 Jun 2013 3,519
13 Wide & Deep Learning for Recommender Systems Heng-Tze Cheng, Levent Koç, Jeremiah Harmsen et al. Conference paperOpen access 15 Sep 2016 3,424
14 Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators Lu Lu, Pengzhan Jin, Guofei Pang et al. · Nature Machine Intelligence Open access 18 Mar 2021 3,345
15 Federated Learning: Strategies for Improving Communication Efficiency Jakub Konečný, H. Brendan McMahan, Felix X. Yu et al. · arXiv (Cornell University) PreprintOpen access 18 Oct 2016 3,043
16 Large Scale Distributed Deep Networks Jay B. Dean, Greg S. Corrado, Rajat Monga et al. 3 Dec 2012 2,903
17 Fast linear iterations for distributed averaging Lin Xiao, Stephen Boyd · Systems & Control Letters 13 Apr 2004 2,789
18 A comparative analysis of gradient boosting algorithms Candice Bentéjac, Anna Csörgő, Gonzalo Martínez-Muñoz · Artificial Intelligence Review Open access 24 Aug 2020 2,730
19 A Reliable Data-Based Bandwidth Selection Method for Kernel Density Estimation Simon J. Sheather, M. C. Jones · Journal of the Royal Statistical Society Series B (Statistical Methodology) Jul 1991 2,706
20 Random Features for Large-Scale Kernel Machines Ali Rahimi, Benjamin Recht 3 Dec 2007 2,633
21 Smooth minimization of non-smooth functions Yu. Nesterov · Mathematical Programming 29 Dec 2004 2,600
22 Understanding deep learning (still) requires rethinking generalization Chiyuan Zhang, Samy Bengio, Moritz Hardt et al. · Communications of the ACM Open access 22 Feb 2021 2,392
23 Federated Learning With Differential Privacy: Algorithms and Performance Analysis Kang Wei, Jun Li, Ming Ding et al. · IEEE Transactions on Information Forensics and Security Open access 2020 2,379
24 Barren plateaus in quantum neural network training landscapes Jarrod R. McClean, Sergio Boixo, Vadim Smelyanskiy et al. · Nature Communications Open access 12 Nov 2018 2,318
25 Adaptive Federated Learning in Resource Constrained Edge Computing Systems Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis et al. · IEEE Journal on Selected Areas in Communications 11 Mar 2019 2,302
26 The Power of Convex Relaxation: Near-Optimal Matrix Completion Emmanuel J. Candès, Terence Tao · IEEE Transactions on Information Theory 26 Apr 2010 2,163
27 Robust Stochastic Approximation Approach to Stochastic Programming Arkadi S. Nemirovski, Anatoli Juditsky, Guanghui Lan et al. · SIAM Journal on Optimization 2009 2,115
28 EIE Song Han, Xingyu Liu, Huizi Mao et al. · ACM SIGARCH Computer Architecture News 18 Jun 2016 2,047
29 Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization P-C Tseng · Journal of Optimization Theory and Applications Jun 2001 2,046
30 Distributed asynchronous deterministic and stochastic gradient optimization algorithms John N. Tsitsiklis, Dimitri P. Bertsekas, Michael Athans · IEEE Transactions on Automatic Control Sep 1986 2,042
31 Optimization flow control. I. Basic algorithm and convergence Steven H. Low, D.E. Lapsely · IEEE/ACM Transactions on Networking 1999 1,982
32 Practical Recommendations for Gradient-Based Training of Deep Architectures Yoshua Bengio · Lecture notes in computer science Conference paper 2012 1,977
33 SecureML: A System for Scalable Privacy-Preserving Machine Learning Payman Mohassel, Yupeng Zhang Conference paper May 2017 1,906
34 Accelerating Stochastic Gradient Descent using Predictive Variance Reduction Rie Johnson, Tong Zhang 5 Dec 2013 1,905
35 Privacy-Preserving Deep Learning via Additively Homomorphic Encryption Le Trieu Phong, Yoshinori Aono, Takuya Hayashi et al. · IEEE Transactions on Information Forensics and Security 29 Dec 2017 1,679
36 Recommender systems with social regularization Hao Ma, Dengyong Zhou, Chao Liu et al. Conference paper Feb 2011 1,664
37 Federated Optimization: Distributed Machine Learning for On-Device Intelligence Jakub Konečný, H. Brendan McMahan, Daniel Ramage et al. · arXiv (Cornell University) PreprintOpen access 8 Oct 2016 1,651
38 Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning Milad Nasr, Reza Shokri, Amir Houmansadr Conference paperOpen access May 2019 1,639
39 Kernel methods in machine learning Thomas Hofmann, Bernhard Schölkopf, Alexander J. Smola · The Annals of Statistics Open access 26 May 2008 1,636
40 Pegasos: primal estimated sub-gradient solver for SVM Shai Shalev‐Shwartz, Yoram Singer, Nathan Srebro et al. · Mathematical Programming 15 Oct 2010 1,526
41 Coordinate descent algorithms Stephen J. Wright · Mathematical Programming 24 Mar 2015 1,494
42 Neural Tangent Kernel: Convergence and Generalization in Neural Networks Arthur Paul Jacot, Franck Gabriel, Clément Hongler · arXiv (Cornell University) PreprintOpen access 20 Jun 2018 1,478
43 The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems E Weinan, Bing Yu · Communications in Mathematics and Statistics 14 Feb 2018 1,471
44 Improved Adam Optimizer for Deep Neural Networks Zijun Zhang Conference paper Jun 2018 1,413
45 Factorization Machines with libFM Steffen Rendle · ACM Transactions on Intelligent Systems and Technology May 2012 1,392
46 Iteratively reweighted least squares minimization for sparse recovery Ingrid Daubechies, Ronald DeVore, Massimo Fornasier et al. · Communications on Pure and Applied Mathematics Open access 19 Oct 2009 1,338
47 Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems Yu. Nesterov · SIAM Journal on Optimization 2012 1,256
48 Efficient projections onto the l 1 -ball for learning in high dimensions John C. Duchi, Shai Shalev‐Shwartz, Yoram Singer et al. Conference paper 2008 1,254
49 Distributed Optimization Over Time-Varying Directed Graphs Angelia Nedić, Alex Olshevsky · IEEE Transactions on Automatic Control 20 Oct 2014 1,232
50 Error bounds for approximations with deep ReLU networks Dmitry Yarotsky · Neural Networks 13 Jul 2017 1,137

Citations are OpenAlex's own counts, from its 2026-09-23 snapshot, so the ranking can differ from Google Scholar or Web of Science. A paper is listed for a field when OpenAlex files it under one of the field's topics; retracted papers and papers whose citation count cannot be traced to citing works are left out, and a preprint and the article it became are shown once. Release dates are OpenAlex's: where a source gave only a month or a year, that is what is shown. How the lists are chosen.