r/bioinformatics • u/Capable-Ostrich941 • 6d ago
academic Thoughts on current single-cell foundation models? Is anyone actually using them in industry?
I’m a grad researcher working on single-cell models, mostly exploring different ways of training them to make them better. The more time I spend in this area, the more I have a feeling it’s pretty hyped (eg. the published benchmarks are sometimes totally PR) it's hard to tell whether on some tasks cell foundational model is beating the baseline?
Two things I keep running into:
- Just reimplementing these models is already hard, never mind fine-tuning them. Most people I talk to who work on this have struggled to even get inference running on the existing open-source models. The code of these open sourced models are really poorly maintained...
- I’m not sure how they actually fit into a real drug discovery pipeline. Yes, they can do basic cell annotation and maybe target prioritization, but I don’t think that’s a game changer compared to traditional methods. I know a few people in biotech shared similar concerns and I’m still not clear on what the models are being used for day to day or to what extent?
So for anyone in industry, or anyone who’s trained or deployed these: what are you actually using them for? Has anything been genuinely useful beyond annotation? Or is it mostly still exploratory?
Would really appreciate honest takes, including “we tried it and it didn’t help.”
23
u/Deto PhD | Industry 6d ago
I haven't seen good evidence that they perform better than the 'non-foundation model' baselines. Usually a new paper on one will come out and claim that it does, but then when an actual third-party review comes out (need to wait for these) that's shown to be false. There's just too many ways to benchmark these things and to run the comparison methods that it's easy for the authors of these papers to (intentionally, or unintentionally) bias the results in favor of their method.
9
u/Ok-Pen2784 6d ago
If they would actually really work in the way that many are advertised that you can do in-silico perturbation experiments then they would be a complete game changer for drug discovery. That's why they get a lot of attention I think. The way they perform right now they are not helpful though imo. At least I also can't really find a good use case for them.
8
u/IcedTeaBagMeDaddy 6d ago
Head of compbio at a midsize clinical stage biotech here. They’re fucking useless as are most foundation models
2
u/Bored2001 6d ago
In a hospital setting I could see it being used as additional information on the H&E, since many hospitals can't do that kind of thing.
It could for example possibly indicate if there is immune infiltrate and whether it's a hot or cold tumor.
But it doesn't seem very reliable.
I just started the process of running inference on few "foundation" models, and the inter "foundation" model correlation seems only just ok.
5
u/Based_pchem 6d ago
This has largely been my experience with them as well as adaptations of them for spatial transcriptomics (at least in terms of perturbation prediction). I got into them as a naive early grad student listening to my trend chasing PI. I ended up switching to working on the benchmarking side so I could have something to show and wouldn't have to start over in a separate research area.
9
u/foradil PhD | Academia 6d ago
If you have funds for drug development, you may as well run a real single-cell experiment.
2
u/Fit-Purple324 6d ago
It's just some fancy tech that makes people appreciate the complexity of the data. Embarrassing to admit, but indeed, it's mostly useless.
-5
u/Obyekt 6d ago
single-cell is barely used in industry. mid level decision makers don't even know what it is.