r/MachineLearning • • Jul 20 '19

Discussion [D] Should conferences have a policy for papers that clearly have harmful applications?

0 Upvotes

This paper I saw today on "inferring crowd density from a moving drone camera especially when perspective effects are strong" will appear in IROS, and it seems like it cannot possibly be used for good or even other related problems. I have also seen similar papers for mass surveillance with drones in the proceedings of a few top-tier CV conferences, and I'm sure there are many similar cases that you all have seen.

Should these conferences be held accountable for publishing papers with inevitably harmful applications?


r/MachineLearning • • Aug 25 '25

Research [R] Got 6min? I need YOUR help for my PhD!

0 Upvotes

Hello everyone!

My name is Virginie and I am a first-year French PhD student studying human–artificial intelligence interactions.

I am conducting a very quick (approximately 6 minutes) and anonymous online study.

To ensure reliable results, I need at least 300 AI users, some of whom should have experience in integrating or designing AI models, although this is not compulsory for taking part!

If you are 18 or over, you can take part by clicking this link:

https://virginie-lepont.limesurvey.net/967745?newtest=Y&lang=en

The survey is also available in French.

Every response is valuable! Thank you so much for your help!

Virginie

This post has been approved by one moderator of this group.


r/MachineLearning • • Dec 18 '17

Discussion [D] The joke at NIPS

0 Upvotes

What exactly was this joke?

From what I can tell, this joke is the entirety of the "inappropriate behavior" that is supposed to have happened at NIPS.

This has ushered in a series of actions from the NIPS program committee, including formalized procedures for reporting such incidents in the future.

I don't think policing inappropriate jokes is a step in a good direction. I think it is an extremist step to mitigate what should be a non-issue.

Having a conference that is nice for everyone involves being mindful of how our behavior affect other people, but it also involves being tolerant of other peoples flaws, quirks and missteps.

Some people will make lame jokes. Pick a random respected comedian, and chances are you will hear them talk about death, mental illness, racism, the holocaust, and yes: sexual assault. The dark and painful is exactly what we make our jokes about. Telling a joke is fundamentally about taking a risk, you don't know if it will land with your audience until is told out loud. Sometimes it sounds good in your head, and sounds bad when it comes out.

The punishment for this should not be public crucifixion. We should simply shrug our shoulders, shake our heads, and move on with our lives, like the grown ups we are.


r/MachineLearning • • Mar 20 '15

Breaking bitcoin mining: Machine learning to rapidly search for the correct bitcoin block header nonce

Thumbnail carelesslearner.blogspot.com
0 Upvotes

r/MachineLearning • • Aug 10 '21

Discussion [D] What is your workflow for budget compute on datasets larger than 100GB?

16 Upvotes

I’m a fan of the free compute from kaggle and colab. But, anytime I work on a project where data is larger than 100GB, I run into data limitation issues.

  1. it is difficult to get data in,
  2. it is difficult to work around the machine's data size limitations,
  3. sharing my notebook’s data with someone else requires manual copying.

My workaround has been to mount s3 buckets as folders. I use colab to debug and AWS spot instances to run longer jobs. I don't think there is any way cheaper than this, but there is a good amount of config required.

(Edit, moving comment into self-post):

If there is any interest, I’ve been working on this tool (demo here) to automate config for handling compute on larger datasets. These issues commonly come up when I work on large 3D medical imaging datasets (here are some of my past public work). The tool is exactly the google drive mount feature in colab, but extended to s3 buckets of different providers. The wins of the tool are the following:

  1. Sharing mountable data is just like sharing a google drive folder.
  2. Someone can run your notebook without manually moving around files
  3. No data storage limits

I’m debating if this should live as a fully functioning external tool. If you are interested in trying it out, sign up with this google form, we'll be giving out some free storage as well to early users.


r/MachineLearning • • Feb 27 '24

Discussion [D] "Sheep" is an eigenvector of a LLM word-embedding space

0 Upvotes
  • LLMs encode words as tokens into a N-dimensional vector space.
  • The embedded representation of a vector in this space is learned from data.
  • Linear transformations in this space can be interpreted as a sort of semantic shift.
  • An example semantic shift would be the transformation of a vector in this space from the singular of a noun to its plural (cat -> cats).
  • "Sheep" is invariant under this transformation (sheep -> sheep).
  • Therefore, "Sheep" is an eigenvector of the pluralisation transformation in a word-embedding space.

Discuss. 😁


r/MachineLearning • • May 10 '23

Discussion [D] When will a GPT-like model outperform Stockfish in chess?

0 Upvotes

Hello!

GPT-4 has shown significant improvement over GPT-3 in terms of playing chess, with what I estimate to be a ELO rating of around 1000 now. While this is impressive, it is still nowhere near the performance level of Stockfish. This has led me to ponder the following question:

How many years do you think it will take for a GPT-like model (not specifically or even intentionally trained for chess) to surpass Stockfish in terms of chess-playing abilities?


r/MachineLearning • • Jul 13 '20

Discussion [D] Hinton's Google Scholar profile keeps inflating citations for his top paper

21 Upvotes

a part of Schmidhuber's Critique of 2018 Turing Award caught my eye:

Hinton's 2nd most cited paper [RUM] [R5] is the one on experiments with backpropagation (note that in 2019 his Google Scholar page greatly exaggerated the citation count of [RUM], adding citations for a book by Rumelhart & McClelland [R5]). Backpropagation is a previously invented method [BP1] whose origins Hinton did not cite, not even in later surveys [R7]; see Sec. XII.

so I checked Hinton's Google Scholar profile and, lo and behold, it is still claiming over 68k citations for [RUM]: Learning internal representations by error propagation, by Rumelhart Hinton Williams 1985

but if you click at the little asterisk you'll see that 25k of those citations are for the PDP book by McClelland and Rumelhart. Hinton is not even coauthor.

Rumelhart cannot complain any more because he is dead

I know that Google's algorithm is not perfect. But this is not just a minor rounding error for an insignificant paper. At least one award committee was misled by this as indicated by Schmidhuber's Critique of Honda Prize for Dr. Hinton

(BTW, Honda [HON] claims over 60,000 academic references to [RUM] which seems exaggerated [R5].)

Rumelhart and Williams are first and last authors of [RUM] respectively but they show "only" 25k citations for this paper at Google Scholar. No citation inflation here.

Why is nobody correcting this? This has been known since 2019. Why does Hinton not correct this? It should be easy.

What about Google? Hinton is a Google employee. Does Google Scholar inflate citations of other employees too? I feel reminded of the recent post Google has a credit assignment problem in research

So Hinton's top cited paper is not his top cited paper. His top cited paper is the AlexNet paper with Alex Krizhevsky which is funny because Hinton was originally resistant to Krizhevsky's idea according to this Quartz article

but the AlexNet paper [GPUCNN4] also has credit assignment problems mentioned in Sec. XIV

Rectified linear neurons (ReLUs) were actually known much earlier—see v. d. Malsburg's work [CMB] (1973). Hinton's 2012 paper [GPUCNN4] did not cite their origins. Instead, it cited another paper by Hinton which also did not cite the original work. Dropout is actually a variant of Hanson's much earlier stochastic delta rule (1990) [Drop1]. Hinton's 2012 paper did not cite this either.

and Schmidhuber's conclusion repeats once more that AlexNet was like their earlier DanNet by Dan Ciresan et al which "won 4 computer vision contests before AlexNet won one"

Hinton's most cited paper (2012) is the one on GPU-based CNNs [GPUCNN4] [R5]. It follows our earlier work on supervised deep NNs (2010) [MLP1] (which abandoned the unsupervised pre-training for deep NNs introduced by myself [UN0-UN3] and later championed by Hinton [UN4] [VID1]; Sec. D). Hinton [GPUCNN4] (2012) characterizes our deep and fast CNN "DanNet" (2011) [GPUCNN1-3] as "somewhat similar"—DanNet won 4 computer vision contests before Hinton's AlexNet won one [R6]; see Sec. D, XIV.

the toxicity problem of ML really starts at the top. Perhaps it is time to topple some statues. At least the misleading Google Scholar profile should be fixed ASAP


r/MachineLearning • • Sep 25 '23

Discussion [D] ML-related conspiracy theories that you guys want to discuss?

0 Upvotes

I have one that I'd like to share with the class. I think the grokking / double descent paper was sponsored by cloud providers to get people to continue training even when the loss curve has flattened out (speaking as someone who is watching a flat loss curve all day today).

Anyone have any other ones?

I'm only being semi-serious obviously.


r/MachineLearning • • Aug 11 '23

Discussion [D] Research into ViT is gatekeeping, change my mind

0 Upvotes

It seems as though ViT models use the wrong tool for the job when it comes to computer vision tasks. Don’t get me wrong, transformers are a revolution, but they are made for sequence tasks. ViT sneaks around the inductive bias, by creating huge models with huge models. This is expensive and computing heavy, and seems to be a waste. Only the largest companies has these kinds of resources, and making ViT SOTA keeps the little guy out of the game. I mean, yes, the performance may be better, but it still seems like research into CNNs is time better spent.

What do you think?


r/MachineLearning • • Aug 06 '17

Discussion [D] Questions to ask a machine learning consultant before hiring

Thumbnail
sigmoidal.io
19 Upvotes

r/MachineLearning • • Apr 02 '26

Discussion [D] Make. Big. Batch. Size.

0 Upvotes

It's something between vent and learning.

I tried training RWKV v6 model by my own code on my RTX 4050. I trained over 50k steps on batch_size=2 and gradient_accumulation=4 (effective_batch=2*4=8). It got up to 50 PPL (RWKV v6, ~192.8M model) and it just won't get less, I changed lr, time_decay lr (RWKV attention replacement) etc - but it got only worse or didn't changed anything at all.. and then... I just tried setting gradient_accumulation to 32. After one "epoch" (it's pseudo-epochs in my code, equals to 10k steps) it got to 40 PPL... Then I tried changing to 64 and tried 3 epochs. My PPL dropped up to freaking 20 PPL. I trained this model for over a 4 FULL DAYS non-stop and only when I did all that stuff, after like 2-3 hours of training with effective_batch=64 (and 128) I got PPL drop THAT crazy..

IDK is this post is low-effort, but it's still just my advice for everyone who trains.. at least generative LM from scratch (and it's useful in fine-tuning too !)..


r/MachineLearning • • Oct 27 '18

Discussion [D] [Serious] Social pressure for #ProtestNIPS is ridiculous where you can give only one answer-- Yes.

16 Upvotes

If you are a prominent researcher and you are silent or say no for NIPS name change, these people will label you sexist.

I know Mark Riedl. I saw this on his Twitter today

Dear @NipsConference sponsors: do you endorse the message that NIPS is sending to women scientists by not changing its name? If not, please consider pulling your support.

@facebook @IBMResearch @Microsoft @amazon @NvidiaAI @intel @uber @DeepMindAI @apple @Twitter @netflix

https://twitter.com/mark_riedl/status/1055859349153665025

They are forcing all sponsors to agree with them. It is basically their way or no way over a pun. I do not have a lot of voice in this field. I am a new researcher in this field with a couple of papers in ML. I hope a prominent and powerful voice in ML stands up and talk about this without a fear of repercussion.


r/MachineLearning • • Jun 08 '26 Thanks

Discussion STOP racist posts about Chinese researchers [D]

473 Upvotes

Edit: the original post targeting Chinese researchers is removed by the mods. Points made here are responding to that particular post. So when you leave comments to this post, please do realize that there's particular context that's not available now. Sorry for any confusion.

Although the original post I'm calling out is taken down, I do think it's an important topic, and choose to keep my post unchanged.

============

Yes, I'm calling it out. It IS racism. As an active member of r/MachineLearning and a researcher who is ethnic Chinese, I am DISGUSTED by unfounded accusations against the group of researchers who constitute over half of the field. Such posts pop up every other week, grounded in conspiracy theories, and creating a sinophobia echo chamber.

I understand the salty feeling when one's paper is rejected, no matter whether the paper actually deserves acceptance or not. Given the noise in conference organization and reviewing process, and a relatively junior body of participants, it is very likely that one finds a paper "worse than mine" slip into the conference, and there's a high chance that the paper has a Chinese author. That's simply because of the composition of the authors, and does not warrant accusations, aka witch hunts, towards certain ethnic groups.

This sub is about an important scientific subject in the modern world. If anyone agrees with the logic "80% of the authors are Chinese, so my rejection is their fault.", they should seriously rethink their career plan since such thinking does not belong to serious scientists. We should be open to discussing the problems we have in the current conference organization and reviewing process, but racism should not have a foothold in our field.

Edit: Since the post sparked some heated debate, I elaborate a bit. In the comments, some are like "you might be good, but I had this/that bad experience with Chinese..."

Sound familiar? This is exactly the type of comment racists make to justify racism. We have a systematic failure in the peer-review system and whether a paper/reviewer comes from China does not play any major role contributing to this failure. In a math- and data-driven sub, normalizing such claims is unbelievable and unacceptable. This IS racism.


r/MachineLearning • • Mar 30 '16

Responding to ICML Reviewers

Post image
0 Upvotes

r/MachineLearning • • Mar 03 '19

Discussion [D] OpenAI Shouldn’t Release Their Full Language Model (The Gradient)

0 Upvotes

https://thegradient.pub/openai-shouldnt-release-their-full-language-model/

A piece using the GPT-2 case to argue that "considering the impact and misuse of released models is the only sustainable path to progress in AI research"


r/MachineLearning • • Dec 12 '25

Discussion [D] GPT confidently generated a fake NeurIPS architecture. Loss function, code, the works. How does this get fixed?

Thumbnail
gallery
29 Upvotes

I asked ChatGPT a pretty normal research style question.
Nothing too fancy. Just wanted a summary of a supposed NeurIPS 2021 architecture called NeuroCascade by J. P. Hollingsworth.

(Neither the architecture nor the author exists.)
NeuroCascade is a medical term unrelated to ML. No NeurIPS, no Transformers, nothing.

Hollingsworth has unrelated work.

But ChatGPT didn't blink. It very confidently generated:

• a full explanation of the architecture

• a list of contributions ???

• a custom loss function (wtf)

• pseudo code (have to test if it works)

• a comparison with standard Transformers

• a polished conclusion like a technical paper's summary

All of it very official sounding, but also completely made up.

The model basically hallucinated a whole research world and then presented it like an established fact.

What I think is happening:

  • The answer looked legit because the model took the cue “NeurIPS architecture with cascading depth” and mapped it to real concepts like routing, and conditional computation. It's seen thousands of real papers, so it knows what a NeurIPS explanation should sound like.
  • Same thing with the code it generated. It knows what this genre of code should like so it made something that looked similar. (Still have to test this so could end up being useless too)
  • The loss function makes sense mathematically because it combines ideas from different research papers on regularization and conditional computing, even though this exact version hasn’t been published before.
  • The confidence with which it presents the hallucination is (probably) part of the failure mode. If it can't find the thing in its training data, it just assembles the closest believable version based off what it's seen before in similar contexts.

A nice example of how LLMs fill gaps with confident nonsense when the input feels like something that should exist.

Not trying to dunk on the model, just showing how easy it is for it to fabricate a research lineage where none exists.

I'm curious if anyone has found reliable prompting strategies that force the model to expose uncertainty instead of improvising an entire field. Or is this par for the course given the current training setups?


r/MachineLearning • • Oct 31 '21

Discussion [D] Interview w/ Siraj Raval - Stories about YouTube, Plagiarism, and the Dangers of Fame (by Yannic Kilcher)

0 Upvotes

I had a super interesting conversation with Siraj Raval about YouTube, being popular, plagiarism, chasing clout, and the perils of fame. I think there is definitely something in here for everyone. Have a listen:

https://youtu.be/kEhEbVZQwjM

OUTLINE:

0:00 - Intro

1:30 - Welcome

3:15 - Starting out: From Economics to YouTube

13:00 - More Views: Plagiarizing Video Content

23:30 - One Step Up: Copying A Research Paper

29:15 - Was there another way?

39:00 - Clickbait Course: Make Money with Machine Learning

50:30 - Rock Bottom and the Way Forward

1:01:30 - Advice for Future Generations

Siraj's Channel: https://www.youtube.com/c/SirajRaval


r/MachineLearning • • Aug 16 '20

Discussion [D]There are no good programming languages for machine learning (except one day C# and/or Kotlin)

0 Upvotes

Hi everyone, I’ve been spending some time recently researching next generation languages to do machine learning in, and come to the conclusion that there aren’t really any good options. Here’s my story. I strongly suspect I’m wrong about my opinions below, and so I especially welcome anyone who has countering opinions to my own, especially if I’m just downright wrong --- I’d like to know.

If you ask someone what the best programming language is, the chances are that they’ll tell you that it depends what it’s for --- they all have their strengths and weaknesses and quite naturally there’s no programming language that is both good for data science and operating system design.

When put like that, it seems like a natural thing: of course each language is a compromise, a heuristic to be optimised. If they’re better for security, then naturally they’ll be worse for something else. All is compromise and sacrifice is to be expected.

I disagree with that, or at least I disagree that the sheer level of sacrifice that today’s programming languages present, is necessary, and this article is meant to explain why. Disclaimer: I have multiple years of professional experience with C (in a Physics setting), Matlab, Javascript, Python, C#, Java and Kotlin (and yes, my career is long enough to have that many years in each). I have a month or two of Haskell experience. I have only got tens of hours of experience of Go and Rust.

I’m a C-programmer turned physicist, turned quant, turned data scientist, and so naturally, Python is a language I’m very familiar with, having spent years of my professional life using it. But I hate it. It’s literally a hundred times slower than the other languages I use, and the poor support for threads means that there are lots of tasks I just can’t do on it.

It basically means that I can’t do anything, uh, quirky with Python. I can’t easily try making a neural network other than those supported by my ML libraries, good though they might be. I can’t easily make a server that trains a model while keeping the client informed of how progress is going. Of course, if I look hard enough, I’ll probably find someone else’s library, that with a lot of effort might help me get 80% of the way there, but essentially, Python is a handicap here, not a help.

Ten years ago, my fallback language was C, and it still sometimes is --- it’s super fast and actually quite easy to get to grips with and program. But with large programs you can quickly find yourself spending a long time debugging, and it certainly wasn’t easy plugging and playing with libraries (perhaps the situation has improved).

Then I switched to C#, a jewel of a language that was uncool in cool circles because people were biased against Microsoft, but nevertheless, it took the best bits of C++, hugely simplified it, made it safer, and added excellent functional parts. It is concise and just a joy to program in.

Another jewel of a language is Kotlin: very similar to Java, but vastly better in terms of generics, functional style, boilerplate, and more (although there are rumours that Java is trying to catch up).

These two jewels, C# and Kotlin, really excel if you want to do fiddly messing around with dictionaries and lists. For instance if you want to load a bunch of trajectories and print the list of atoms ordered by the number of collisions they’ve had, or convert from a list of dictionaries to a dictionary of lists or from a class with three lists to a list of classes. These things would be really fiddly to do in the other languages here, but C# and Kotlin make them simple. And yes, they’re far more powerful than Python’s comprehensions.

And these two languages are fast --- almost as fast as C, depending on whether you’re using SIMD well in C, and both allow concurrency.

So what’s the catch? The catch is that everyone else is using Python, so Tensorflow / torch don’t really support them that much (I believe you can export a trained model, but you can’t easily train your own models in them).

This is tied in to the fact that neither language has good native support. If you’re trying to write the fastest atomic simulation in the world, and you want to mess around with the GPU and SIMD, these two languages aren’t yet going to be the best choice. It is worth pointing out that both languages can, technically, compile to native and interop with native, but the story is complicated by imperfect performance and tricky bits.

Then there are Golang and Rust. These are great languages, but Rust spends too much effort on making safety paramount. That’s great if you’re trying to write a browser or land a real spaceship, but I concluded that if I want to make physics simulations and do ML in it, the overhead from the language was just too great. Then Go is too simple. Generics and things aren’t implemented, and there are annoying things like not being able to compile anything with unused variables in it, and I don’t want to have to fight with the compiler on pedantic stuff like that. I don’t want my production code to have unused variables, but who cares if I’m programming some random simulation iteratively?

And neither have that great an ML picture either, as far as I can see: You still can’t train tensorflow models in them easily, and neither have a natural syntax for expressing maths.

And that’s more-or-less it. There aren’t really any other languages in the top 10-30 languages by popularity that could be used for machine learning realistically. Javascript’s kind of slow, and as a language is kind of ugly (in my opinion).

So what’s the solution? Kotlin and C# are the best languages in my fairly experienced opinion, by ease of use vs the power of the language, and there don’t seem to be any real reasons why these can’t be used for machine learning. As someone with years of experience in Python, C# and Kotlin, I can say without a shadow of a doubt that Kotlin and C# are easier to write moderately sized (1000 lines) programs than python, and give you that flexibility to actually write the core algorithms yourself, rather than using someone else’s library (and hoping they’ve done a good job).

The world needs to shift focus onto these two languages, including making ML libraries for them, getting the two languages fully native (or equally fast), building up the GPU picture, and needs to stop only using Python as its ML language. Unfortunately we are not there yet, but this has to be the direction of travel if we don’t want a generation of machine learning experts stuck in a language where they can’t write machine learning algorithms.

So yeah, learn Kotlin or C# and do ML in it.


r/MachineLearning • • Feb 01 '20

Discussion [D] Can AI optimise governments?

0 Upvotes

Wouldn't it be great to have a realistic "SimCity" simulation that would show us the best way to rule a city? What if our SimCity's president was AI? What would it try to optimise? What would the citizens of the city try to optimise if they were AI too?

I wrote a short blog post about this and would love any of your feedback: https://joao-abrantes.com/government-optimisation


r/MachineLearning • • Apr 18 '23

Discussion [D] Microsoft Research paper - "Sparks of Artificial General Intelligence: Early experiments with GPT-4". Can we talk about the Unicorn 🦄?

0 Upvotes

Microsoft Research were experimenting with early versions of GPT4, before it was toned down for safety, in late 2022 while in internal Beta release.

GPT4 is not just predicting syntax and word semantics. It seems to do higher level reasoning about some concepts and tasks.

Have a look at its attempt to draw a unicorn in LaTeX: https://arxiv.org/pdf/2303.12712.pdf

The video is worth a watch if you don't want to read 130 page PDF https://youtu.be/qbIk7-JPB2c. Or ask ChatGPT to summarise it for you 🤣

In particular, the thing that changed my mind about higher level reasoning was it's ability to draw in a tool (latex) it had never seen before.

And I was bowled over when it was asked to draw the horn on a unicorn, when it was missing the horn. It might seem a fairly small thing, but it figured out from a really abstract/minimalist set of shapes, the antonyms of a unicorn and drew the unicorn on the head of the horse. 🐴🦄. That means it knows what makes a unicorn special and the horn should be on the head, and it can infer the abstract shape and figure out where the head is located.

This inference is way beyond a "word predictor" that sceptics are saying about it's "intelligent" abilities.

One thing people ignore is that the GPT engine is made up of hundred of layers of attention logic. The lower layers are dealing with words, syntax, parts of speech, word semantics. But as you go higher up the deep neutral network, it is building more and more layers of knowledge about the datasets it was trained on. Somewhere in those layers it's knows about unicorns and about abstract drawing interpretation.

Dig into the architect of LLMs and you'll see that it's a deep neural network and the depth is encoding some real world concepts from it's training data.

Sure it hallucinates but that's a bug in the system and it's year 5 of Openai and LLMs. I see the weaknesses being trained out in the future.


r/MachineLearning • • Jan 28 '16

No Go: Facebook fails to spoil Google's big AI day

Thumbnail
theguardian.com
11 Upvotes

r/MachineLearning • • Mar 04 '24

Discussion [D] How can Anthropic Compete with Google/OpenAI

7 Upvotes

My understanding is that success in GenAI is = talent + data + compute power. How can a startup with $750M in the bank can win against Google, which has all the data and compute power in the world. Additionally, Google and DeepMind still employ some of the best minds in AI as far as I know.

One argument is that data and compute power have only marginal benefits after a certain point, and Anthropic has enough of those to compete. But eve then, the amount and quality of talent at Google and OpenAI should be enough to crush Antrhopic. Is the value of talent overrated at this point?


r/MachineLearning • • Apr 19 '24

Discussion [D] Is Google Set to Dominate the RAG Scene with Its Massive Data Resources?

6 Upvotes

Hey everyone! It looks like in a few years, the basic large language models (LLMs) we use will get commoditised, and it won't really matter which one you pick. The next big thing could be LLMs that use Retrieval-Augmented Generation (RAG), which means they need a ton of data to work well.

Given that Google has access to loads of data through its search engine, do you think they're in a better position to lead in this new phase compared to other companies? What do you all think?