Science Explorer Back to mapMap
Discussion
Topic · Artificial Intelligence

Stochastic Gradient Optimization Techniques

Stochastic Gradient Optimization Techniques is a research topic within Artificial Intelligence. Science Explorer counts 7.9k research works in it since 1965. 10.1% of them reached the world's top 10% most cited for their field and year.

This cluster of papers focuses on the application of optimization methods in machine learning, particularly in the context of stochastic gradient descent, random projections, deep learning, convex optimization, matrix decompositions, and large-scale optimization. The papers explore various algorithms and techniques for improving the efficiency and effectiveness of machine learning models, with a specific emphasis on neural networks and generalization.

  • Stochastic Gradient Descent
  • Random Projections
  • Deep Learning
  • Convex Optimization
  • Matrix Decompositions
  • Approximation Algorithms
  • Large-Scale Optimization
  • Neural Networks
  • Coordinate Descent
  • Generalization
Research works
7.9k
fractional, since 1965
In the world top 10%
799
per year above
Top-10% rate
10.1%
share of its works in the world top 10%
Growth, 2018-21 → 2022-25
+76%
the tick is no change
Largest producer
China
45% of the 10 largest, 2022-now
Stochastic Gradient Optimization Techniques

Which countries lead Stochastic Gradient Optimization Techniques research?

By volume, China and the United States publish the most (1.6k and 970 works in 2022-now).

By volume, 2022-now

  1. 1 China 1.6k works
  2. 2 United States 970 works
  3. 3 India 169 works
  4. 4 Germany 133 works
  5. 5 United Kingdom 123 works
  6. 6 France 121 works
  7. 7 South Korea 105 works
  8. 8 Japan 104 works
  9. 9 Canada 102 works
  10. 10 Italy 88 works

How concentrated that is

The same countries as shares of everything the list above accounts for. A node where two countries do two thirds of the work and one spread evenly across twelve read alike as a ranking and not at all alike here.

China: 44.9%United States: 27.9%India: 4.9%Germany: 3.8%6 others listed: 18.5%45%largest
China1,564 · 44.9%United States970 · 27.9%India169 · 4.9%Germany133 · 3.8%6 others listed644 · 18.5%

Shares of the rows listed above, not of the whole node.

Which institutions lead Stochastic Gradient Optimization Techniques research?

By volume in 2022-now, Chinese Academy of Sciences publishes the most Stochastic Gradient Optimization Techniques research, followed by Centre National de la Recherche Scientifique and Shanghai Jiao Tong University.

By volume, 2022-now

  1. 1 Chinese Academy of SciencesChina 51 works
  2. 2 Centre National de la Recherche ScientifiqueFrance 38 works
  3. 3 Shanghai Jiao Tong UniversityChina 36 works
  4. 4 Tsinghua UniversityChina 34 works
  5. 5 Sun Yat-sen UniversityChina 33 works
  6. 6 Peking UniversityChina 31 works
  7. 7 Zhejiang UniversityChina 29 works
  8. 8 Hong Kong University of Science and TechnologyChina 28 works
  9. 9 Georgia Institute of TechnologyUnited States 28 works
  10. 10 Beijing University of Posts and TelecommunicationsChina 27 works

Who are the leading researchers in Stochastic Gradient Optimization Techniques?

The most-cited researchers publishing on Stochastic Gradient Optimization Techniques include Yoshua Bengio, Geoffrey E. Hinton and Robert John Tibshirani.

  1. 1 Yoshua Bengio Canada 476k citations
  2. 2 Geoffrey E. Hinton Canada 454k citations
  3. 3 Robert John Tibshirani United States 396k citations
  4. 4 Xiangyu Zhang 327k citations
  5. 5 Trevor Hastie United States 301k citations
  6. 6 Yann LeCun United States 256k citations
  7. 7 Andrew Zisserman United Kingdom 255k citations
  8. 8 Mark Gerstein United States 250k citations
  9. 9 Li Fei-Fei United States 225k citations
  10. 10 Kai You Wang China 224k citations

Ranked by citations received across their whole record, among researchers with at least three works on this topic.

What are the most cited papers in Stochastic Gradient Optimization Techniques?

The most cited paper in Stochastic Gradient Optimization Techniques is "Adam: A Method for Stochastic Optimization" (Diederik P. Kingma, Jimmy Lei Ba, 2014), with 84,625 citations. Next come "Adam: A Method for Stochastic Optimization" (50,327) and "Adaptive Subgradient Methods for Online Learning and Stochastic Optimization" (8,604).

Most cited papers
#PaperReleasedCitations
1 Adam: A Method for Stochastic Optimization Diederik P. Kingma, Jimmy Lei Ba · arXiv (Cornell University) PreprintOpen access 22 Dec 2014 84,625
2 Adam: A Method for Stochastic Optimization Mandi, Jayanta, Canoy, Rocsildes, Bucarey, Víctor et al. · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) Open access May 2015 50,327
3 Adaptive Subgradient Methods for Online Learning and Stochastic Optimization John C. Duchi, Elad Hazan, Yoram Singer 2010 8,604
4 Deep Learning with Differential Privacy Martı́n Abadi, Andy Chu, Ian Goodfellow et al. Conference paperOpen access 24 Oct 2016 6,312
5 Large-Scale Machine Learning with Stochastic Gradient Descent Léon Bottou Book chapter 2010 5,757
All 100 most cited papers, with filters and sorting

Where is Stochastic Gradient Optimization Techniques research done?

The largest centres of Stochastic Gradient Optimization Techniques research in 2022-now are Beijing (China), Shanghai (China), Nanjing (China) and Guangzhou (China). Among places with at least 20 works in it, it is an unusually large share of all research in West Lafayette, Pok Fu Lam and Hong Kong.

Largest cities, 2022-now

  1. 1 BeijingChina 290 works
  2. 2 ShanghaiChina 133 works
  3. 3 NanjingChina 92 works
  4. 4 GuangzhouChina 80 works
  5. 5 Hong KongChina 78 works
  6. 6 ShenzhenChina 73 works
  7. 7 Xi'anChina 67 works
  8. 8 SeoulSouth Korea 64 works
  9. 9 HangzhouChina 62 works
  10. 10 ChongqingChina 58 works

Where it is the local speciality

  1. West LafayetteUS · 27.0 works6.9×
  2. Pok Fu LamCN · 23.2 works6.5×
  3. Hong KongCN · 77.7 works5.7×
← less than its size predicts1×more →

Location quotient: how much more of its research is in Stochastic Gradient Optimization Techniques than the world average.

See Stochastic Gradient Optimization Techniques on the map

Where is the best place to study Stochastic Gradient Optimization Techniques?

Among universities, judged by research, Hong Kong University of Science and Technology, Nanyang Technological University and Tsinghua University score highest, combining excellence, specialisation, size, growth and international reach. Research strength is one signal when choosing where to study; it does not measure teaching.

0%10%20%mean 15.38%fractional works in this field (log) →share of work in the world top 10% →Nanyang Technologica…Hong Kong University…Tsinghua UniversityUniversity of Scienc…
above the meannear itbelow it

One dot per university in the first ten of the table below, at the default weights. The upper left is the interesting corner: small places doing unusually strong work.

Excellenceshare of its work in the world top 10%, 2022-now and all time averaged
40%
Specialisationhow much of its work is in this field, counted up to 10× the world share
10%
Sizeits output in this field, on a log scale
30%
Growthoutput growth, counted from -20% to +20%
10%
Internationalshare of its work with a co-author abroad
10%
Each slider sets a weight; the share it gets of the score is beside it.
#UniversityScoreTop 10%SpecialisationWorksGrowth
1Hong Kong University of Science and TechnologyChina 86.318.6%18.6×28 +261.5%
2Nanyang Technological UniversitySingapore 81.822.7%6.6×19 +87.7%
3Tsinghua UniversityChina 74.216.5%4.0×34 +37.3%
4University of Science and Technology of ChinaChina 70.314.9%6.0×26 +75.3%
5Sun Yat-sen UniversityChina 70.214.6%4.3×33 +162.1%
6University of Hong KongChina 68.511.4%6.8×23 +65.9%
7Peking UniversityChina 67.812.8%4.1×31 +63.9%
8Beijing University of Posts and TelecommunicationsChina 66.012.8%11.5×27 +368.3%
9University of Electronic Science and Technology of ChinaChina 65.214.6%5.6×23 +207.8%
10Hong Kong Polytechnic UniversityChina 65.214.9%4.9×16 +236.0%

Universities only. The score blends excellence (40%), size (30%), specialisation (10%), growth (10%), international reach (10%), 2022-now; growth compares 2018-21 with 2022-25 and counts up to +20%, since a jump from a small base is mostly noise.

Every institution in Stochastic Gradient Optimization Techniques, country by country

Is Stochastic Gradient Optimization Techniques research growing?

Output in 2022-2025 was 76% higher than in 2018-2021, peaking in 2026. The fastest-growing topics are Stochastic Gradient Optimization Techniques.

still counting19801990200020102020
grewheldshrank

The same series as a ribbon: one cell per year, darker for more. The line above answers how much; this answers when.

Its share of all research

Growing output can still be a shrinking share: all of science grows. This is Stochastic Gradient Optimization Techniques as a percentage of every work published that year, 0.014% in 2024.

still counting0.010%0.020%Stochastic Gradient Optimization Techniques0.018% in 202519801990200020102020

Fractional works of Stochastic Gradient Optimization Techniques over all fractional works that year. How the balance of science has shifted.

How much of it reaches the world top 10%

Year by year.

still counting5%10%15%Stochastic Gradient Optimization Techniques10.3% in 202520102020

Of each year's works in it, the share that reached the world's 10% most cited for their field and year. A field is the world for its own topics, so there is no other line to hold it against.

Which topics inside it are moving

Growth and decline on one axis around a shared zero. Two lists side by side hide the thing that matters: whether the growth dwarfs the decline, or the other way round.

Discussion

Ask a question or share a note about Stochastic Gradient Optimization Techniques. Posts are plain text and can be reported.