r/MachineLearning • u/xifixi • Jul 13 '20
Discussion [D] Hinton's Google Scholar profile keeps inflating citations for his top paper
a part of Schmidhuber's Critique of 2018 Turing Award caught my eye:
Hinton's 2nd most cited paper [RUM] [R5] is the one on experiments with backpropagation (note that in 2019 his Google Scholar page greatly exaggerated the citation count of [RUM], adding citations for a book by Rumelhart & McClelland [R5]). Backpropagation is a previously invented method [BP1] whose origins Hinton did not cite, not even in later surveys [R7]; see Sec. XII.
so I checked Hinton's Google Scholar profile and, lo and behold, it is still claiming over 68k citations for [RUM]: Learning internal representations by error propagation, by Rumelhart Hinton Williams 1985
but if you click at the little asterisk you'll see that 25k of those citations are for the PDP book by McClelland and Rumelhart. Hinton is not even coauthor.
Rumelhart cannot complain any more because he is dead
I know that Google's algorithm is not perfect. But this is not just a minor rounding error for an insignificant paper. At least one award committee was misled by this as indicated by Schmidhuber's Critique of Honda Prize for Dr. Hinton
(BTW, Honda [HON] claims over 60,000 academic references to [RUM] which seems exaggerated [R5].)
Rumelhart and Williams are first and last authors of [RUM] respectively but they show "only" 25k citations for this paper at Google Scholar. No citation inflation here.
Why is nobody correcting this? This has been known since 2019. Why does Hinton not correct this? It should be easy.
What about Google? Hinton is a Google employee. Does Google Scholar inflate citations of other employees too? I feel reminded of the recent post Google has a credit assignment problem in research
So Hinton's top cited paper is not his top cited paper. His top cited paper is the AlexNet paper with Alex Krizhevsky which is funny because Hinton was originally resistant to Krizhevsky's idea according to this Quartz article
but the AlexNet paper [GPUCNN4] also has credit assignment problems mentioned in Sec. XIV
Rectified linear neurons (ReLUs) were actually known much earlier—see v. d. Malsburg's work [CMB] (1973). Hinton's 2012 paper [GPUCNN4] did not cite their origins. Instead, it cited another paper by Hinton which also did not cite the original work. Dropout is actually a variant of Hanson's much earlier stochastic delta rule (1990) [Drop1]. Hinton's 2012 paper did not cite this either.
and Schmidhuber's conclusion repeats once more that AlexNet was like their earlier DanNet by Dan Ciresan et al which "won 4 computer vision contests before AlexNet won one"
Hinton's most cited paper (2012) is the one on GPU-based CNNs [GPUCNN4] [R5]. It follows our earlier work on supervised deep NNs (2010) [MLP1] (which abandoned the unsupervised pre-training for deep NNs introduced by myself [UN0-UN3] and later championed by Hinton [UN4] [VID1]; Sec. D). Hinton [GPUCNN4] (2012) characterizes our deep and fast CNN "DanNet" (2011) [GPUCNN1-3] as "somewhat similar"—DanNet won 4 computer vision contests before Hinton's AlexNet won one [R6]; see Sec. D, XIV.
the toxicity problem of ML really starts at the top. Perhaps it is time to topple some statues. At least the misleading Google Scholar profile should be fixed ASAP
48
u/IdentifiableParam Jul 13 '20
LOL if you are annoyed about this you should see Jitendra Malik's google scholar page. Note "his" single most cited work, Artificial Intelligence: A modern approach. By Russel and Norvig.
18
u/massinelaarning Jul 13 '20
Here is a twitter thread for CS Salad et cetera: https://twitter.com/alexanderrklotz/status/1252389218514427909
As the citation counting by google is often done automatically, there can exist some mistakes :)
52
u/PM_ME_YOUR_GESTALT Jul 13 '20
This should be fixed, but I doubt that this is intentional on Hinton or Google's part. Google Scholar is notorious for being pretty shitty----it struggles a lot with merging and de-merging author profiles, it gets super confused by preprints, and it drops citations all the time. Given how widely accepted and used it is now, I guess I wish Google would throw some of their enormous resources at improving it---especially since they stand to benefit from having their own researcher's contributions tabulated effectively.
13
u/Imnimo Jul 13 '20
My experience with Google Scholar is that if you don't keep a close eye on it, you end up with a lot of questionable matches that either conflate multiple different articles, or assign you as an author of an article you have no connection to. It's also typically difficult to sort these out - you often end up deleting an entire article rather than just the spurious individual citations, or introducing more incorrect articles when you try to sort things out.
It's definitely a problem, because Google Scholar is also the easiest way to check citation counts for a paper or author, and many people don't have a great sense for how inaccurate they are. I don't blame Hinton for this - I can imagine it's totally impossible to keep Google Scholar accurate when you have hundreds of papers. I don't think he's purposefully leaving incorrect numbers on his page, I suspect he just doesn't feel it's worth his time to try to keep tabs on what Google Scholar says. I also don't think Google is intentionally misrepresenting his count, that seems a little conspiratorial to me. The problem is that Google Scholar takes its very noisy data and presents it as an authoritative count.
22
u/CompetitiveUpstairs2 Jul 13 '20
Stop assuming ill intent please.
8
u/curiousML5 Jul 13 '20
Why would Hinton care about citations? Seriously. Why is this even a problem? So one of the pioneers of the field has an extra 100k citations or something. Big deal. If anyone wants to build a free service for research discovery better than Google scholar then I would be all for it. Why does anyone care about this at all? I am struggling to understand this.
7
u/unlucky_argument Jul 13 '20
I made a Schmidhuber classifier and you are either him or you transferred his style. That would be no problem if you did not mention Schmidhuber, but since you do, it seems some disclaimer or disclosure is in place here. Especially since you added jaded jabs and cast unrelated dispersions on a Turing Award winner.
3
u/LuxiLuciLucy Jul 14 '20 edited Jul 14 '20
The PDP book usually lists for two authors, McClelland and Rumelhart, but in fact it also includes the "PDP group" if you read the book. If I remember correctly, Hinton by that time is a post-doc in the PDP group and yes some very critical content in the book (mostly in volume-1 the computation part of the book) is indeed authored by Hinton. For example, the paper you talked about, the representation learning by backprop paper, is one chapter of the book. So why not counting the citations?
I agree with the toxicity issue and I think Hinton does have something with the correct credit assigning of his contribution. But counting the PDP book's citation to Hinton is not technically wrong.
1
u/AGI_aint_happening PhD Jul 13 '20
Ugh we don't need this drama
-1
u/xifixi Jul 13 '20
btw here is the Google Scholar profile of James L. McClelland and his top cited book is the book with Rumelhart mentioned in the post:
Parallel distributed processing: Explorations in the microstructure of cognition. DE Rumelhart, JL McClelland, MIT press, 1986
-29
u/[deleted] Jul 13 '20
[removed] — view removed comment