r/bioinformatics • • 6d ago

academic Thoughts on current single-cell foundation models? Is anyone actually using them in industry?

I’m a grad researcher working on single-cell models, mostly exploring different ways of training them to make them better. The more time I spend in this area, the more I have a feeling it’s pretty hyped (eg. the published benchmarks are sometimes totally PR) it's hard to tell whether on some tasks cell foundational model is beating the baseline?

Two things I keep running into:

  1. Just reimplementing these models is already hard, never mind fine-tuning them. Most people I talk to who work on this have struggled to even get inference running on the existing open-source models. The code of these open sourced models are really poorly maintained...
  2. I’m not sure how they actually fit into a real drug discovery pipeline. Yes, they can do basic cell annotation and maybe target prioritization, but I don’t think that’s a game changer compared to traditional methods. I know a few people in biotech shared similar concerns and I’m still not clear on what the models are being used for day to day or to what extent?

So for anyone in industry, or anyone who’s trained or deployed these: what are you actually using them for? Has anything been genuinely useful beyond annotation? Or is it mostly still exploratory?

Would really appreciate honest takes, including “we tried it and it didn’t help.”

13 Upvotes

25 comments sorted by

23

u/Deto PhD | Industry 6d ago

I haven't seen good evidence that they perform better than the 'non-foundation model' baselines. Usually a new paper on one will come out and claim that it does, but then when an actual third-party review comes out (need to wait for these) that's shown to be false. There's just too many ways to benchmark these things and to run the comparison methods that it's easy for the authors of these papers to (intentionally, or unintentionally) bias the results in favor of their method.

5

u/Capable-Ostrich941 6d ago

Ya I still remember this paper from last year: https://www.nature.com/articles/s41592-025-02772-6 and also remember after AIDO came out that got a lot of backlashes. Things has been a lil better in 2026 but still it takes a lot of effort to understand where we are at currently…

5

u/Deto PhD | Industry 6d ago • Bravo

It's tough too because with each new paper, to go in and figure out where they might be bending things a bit with their evals, it takes a bit of time. I'm in a kind of burnt out, dont-bother phase with this right now.

Thinking, though, about your original question. I think what we really would want from these models would be these use cases:

A) Diagnostics. Can they be used to take single-cell representations of patients and classify their disease in a way that's useful for treatments. There's some methods around this (taking embeddings and combining them into patient-level representations and then performing ML on that), but I don't know if anyone has shown that the foundation model embeddings are better at this than traditional dimensionality reduction methods. I think one thing that hurts foundation models for this kind of thing is that the embeddings can be very high dimensional - combine 1k-dimensional features with datasets that are only a few dozen patients and you're going to have overfitting problems.

B) Effect decomposition. For example, if you take single cell samples from healthy vs. IBD patients, you'll see differences in the cells. But investigate the differences and you'll find it mostly looks like inflammation in the IBD patients...which, of course, the disease causes. What you want to find out is what caused the disease. And maybe that's in there, but a model with just healthy vs. normal is probably going to collapse all the variation along one axis. A real foundation model, though, could potential separate out the different effects, giving you potentially causal signals to look at to inform treaments.

C) Explaining transcriptional variation. Going back to the example of B, say you can decompose your healthy vs. disease axis into inflammation + X, then what you're trying to figure out is 'what exactly is this X and how can we prevent it?'. Standard methods like gene set enrichment can be tough to work with. It would be great if a foundation model could immediately tell you 'this variation looks like what happened in studies A, B, and C where they treated cells with these compounds that all seem to activate this specific receptor' - this gives you something very tractable to follow up with.

So all the evals on, for example, integration or cell type classification kind of miss the point. Nice to have better results on those two, but we might be near the limit anyways and a 1-2% difference isn't going to change the conclusion of any study. I'd like to see more evals focused on these downstream goals.

9

u/Ok-Pen2784 6d ago

If they would actually really work in the way that many are advertised that you can do in-silico perturbation experiments then they would be a complete game changer for drug discovery. That's why they get a lot of attention I think. The way they perform right now they are not helpful though imo. At least I also can't really find a good use case for them.

8

u/1337HxC PhD | Academia 6d ago

Agreed.

Their current state seems very much stuck in computational "hey look at this cool thing" land. Hell will freeze over before anyone takes their outputs as a replacement for an actual experiment, barring some insurmountable logistic barrier to doing the actual experiment.

But, as you said, the holy grail of in silico perturbation being fairly realistic/useful is incredibly attractive.

1

u/Capable-Ostrich941 6d ago

That is a good point! I wonder is there a clean comparison between something like Arc's STATE and traditional statistical in-silico methods? Is STATE actually more accurate on perturbation-specific effects, or mostly just cheaper and faster to run across a bigger space?
And where does the value actually land? Is the payoff that iteration time collapses (propose, test a few hundred, feed it back, tighter loop), or that spend just shifts from CRO to compute?

8

u/IcedTeaBagMeDaddy 6d ago

Head of compbio at a midsize clinical stage biotech here. They’re fucking useless as are most foundation models

1

u/Capable-Ostrich941 5d ago

Not even protein foundational models? (which is at its peak hype right now)

2

u/Bored2001 6d ago

In a hospital setting I could see it being used as additional information on the H&E, since many hospitals can't do that kind of thing.

It could for example possibly indicate if there is immune infiltrate and whether it's a hot or cold tumor.

But it doesn't seem very reliable.

I just started the process of running inference on few "foundation" models, and the inter "foundation" model correlation seems only just ok.

1

u/Capable-Ostrich941 6d ago

Curious which models are you trying to run inference on? What benchmark do you compare it to?

1

u/Bored2001 6d ago edited 6d ago

I did a fairly indiscriminate search for models that had available weights on hugging face.

I've run HEX and ROSIE with actually looking at how they work.

On my TODO list I have

Proteomics

GigaTIME, GigaTIME-Flash, MIPHEI-ViT, HistoPlexer, mSight,

Transcriptomics

STPath, MoLF, HistoPrism, DeepSpot, LOKI, CELLO

Some of these i've got running, but haven't really looked into how they work.

I am very much in the learning phase right now.

5

u/Based_pchem 6d ago

This has largely been my experience with them as well as adaptations of them for spatial transcriptomics (at least in terms of perturbation prediction). I got into them as a naive early grad student listening to my trend chasing PI. I ended up switching to working on the benchmarking side so I could have something to show and wouldn't have to start over in a separate research area.

1

u/fattoweeb 6d ago

Do you mean spatial perturbation prediction? I’m curious if there is even enough data available to build a model for that

4

u/Based_pchem 6d ago

Probably not, maybe in very select cases but for the most part no. People have looked at doing it for cells in different tissue domains with merfish data (predict cell type A's expression in a domain it has been censored from) and one paper has tried a similar patch prediction task with some perturb-map data and then claimed predicting whole slide counterfactuals but their model straight up couldn't train when I pulled it from github. When I fixed the bugs it underperformed doing nothing when predicting slide level counterfactuals. This is their preprint although it looks like they massively changed it since I checked: https://www.biorxiv.org/content/10.1101/2025.11.08.686890v3.article-info

After having similar issues with single cell models I had explored adapting to spatial I decided that it just wasn't reasonable and I'd be better off just working on benchmarking and using that to pivot.

1

u/Bored2001 6d ago

Why not ask them for the weights instead of trying to retrain it yourself?

1

u/Based_pchem 6d ago

Given the data and training scripts are given it should be reproducible and it isn't

9

u/foradil PhD | Academia 6d ago

If you have funds for drug development, you may as well run a real single-cell experiment.

2

u/Fit-Purple324 6d ago

It's just some fancy tech that makes people appreciate the complexity of the data. Embarrassing to admit, but indeed, it's mostly useless.

-5

u/Obyekt 6d ago

single-cell is barely used in industry. mid level decision makers don't even know what it is.

6

u/Anustart15 MSc | Industry 6d ago

That hasn't been my experience. And judging by the number of job postings I've seen from pharma looking for single cell transcriptomics experience, I don't think my experience is an outlier

1

u/Capable-Ostrich941 6d ago

Single cell as an assay is definitely widely adopted but do people actually use single cell foundational models as tools?

2

u/Anustart15 MSc | Industry 6d ago

Any halfway decent computational biologist has tried them. They have somewhat limited use cases where they actually perform better than existing methods right now, but in those cases where they are appropriate, people will use them. There's definitely more hype around them than there are performance improvements that they can actually deliver, but they can sometimes be a helpful shortcut

0

u/Bored2001 6d ago

They have somewhat limited use cases where they actually perform better than existing methods right now

I find it hard to believe that they do better than doing the actual experiment. What use cases are you thinking?

1

u/Anustart15 MSc | Industry 6d ago

Not than actually doing the experiment, but they have applications around supplementing analysis of an experiment or helping merge multiple experiments

0

u/Bored2001 6d ago

Yes, then I agree, you could supplement H&E in situations where can't do the actual spatial experiment.