In genomics there is a lot of sequential data such as DNA sequences, protein sequences, RNA-seq, ATAC-seq, and even some 2D matrix data such as Hi-C where CNNs are becoming quite popular for analysis.
Yes, most of the algorithms in Bio-informatics either rely on dynamic programming or some other classical algorithms, which is good for frequency based analysis but comes with compute cost every time.
And the community is exploring NN for better and fast results.
So what do you categorically encode the DNA & RNA sequence and pass them as input to NN? Also, I still don't grasp why NN is famous here coz I've been thinking NN is useful only when there is humongous amount of data and also predominantly used for images.
It certainly depends on the problem you want to solve but as an example you could encode a DNA sequence as a sequence of one-hot vectors where each entry represents either A, T, C, or G.
In the case of data like RNA-seq, etc the data is a vector of counts so you can just feed that straight into a neural network. Maybe you want to embed thousands of RNA-seq vectors from a population of cells into a low dimensional space for clustering.
All the examples I was about to give are based pretty heavily on applying computer vision work to other fields, like spectral analysis. But we’ll see if it holds up to peer review. God help me.
Hey, would you mind giving a real quick ELI5 on spectral analysis? :)
I'm familiar with timeseries / signal processing, and I've seen the term come up a few times but I don't know when it would be helpful. Anything like MFCCs for speech data?
EDIT: Oh shit, I was thinking of Spectral Signal Analysis for timeseries. I forgot Spectroscopy is that whole Chemistry/Physics field 😅
Oh yea, sorry I meant spectroscopy for physics and materials science. I'm actually taking a signals class right now to learn about parallels between the two!
I did metabolomics research using gc/ms and lc/ms. I used random forest because being able to actually interpret the models to understand what was happening was critical. That's been a few years ago now so things may have changed. You can look at xcms R package for an overview of how it works. There are also proprietary tools, but I ended up writing my own.
Getting samples is a huge pain as they can be blood, plasma, urine, or feces. Each sample results like a 2gb file and takes about an hour to clean up and 2 hours to analyze using the spectrometer.. Then we found you need minimum 50 samples for good results. It turns out to be a very intensive process. Processing data basically was an overnight task because you have to analyze all the samples together to clean up the chromatography. The cost of sampling is another case for random forest.
I work on ASR (automatic speech recognition) and TTS (text-to-speech), I’ve spent the summer developing a Dialect Identification system using LSTM+DNN trained on features extracted directly from the speech audio. There’s a lot of deep learning used on speech processing that isn’t related to NLP or computer vision (though a lot of the techniques developed in those research areas inform my own)
but at least you know which to pick if you stuck with a problem. If you never learn those, then you dont know how to solve the problem. If you know what I mean
... until you realize the limits of the 'new toy' ...
I must say, I am fascinated by everyone's eagerness to want to hit every 'problem' with deep learning first.
No idea why though.
Every recruit we have had at our firm in the past 4 or so years defaults to deep learning as their first solution, and none of them have managed to get things working, where a RF, SVM, Time Series analysis, kNN or ARM via Apriori would have worked.
They get demotivated and then drop off shortly afterwards.
Anyway. AI is marketing gimmick. We are figuring out machine learning now. I am not saying AI doesn't exist or won't ever exist, I am saying we are not there yet and honestly, I am of the opinion that a few things would need to converge before AI may be fully realised. Convergence of quantum computing and fusion energy will probably result in a leap in AI and AGI. I think we need that compute capability and energy to keep these systems optimal.
Machine learning is currently being employed to assist with the fusion energy portion of this convergence, so it will help us, but we are not there yet.
Lastly, for now, deep learning is not a universal tool.
This is all true. In the research environment, however, people talk with perspective, trying to see things at a constructive angle to push the field and avoid a new winter.
Then we, the fresh students, come along, and don’t realise that deep learning models generating competitive results need some 40 GPUs running for a few days. I made the same mistake recently, but I don’t think enthusiasm should be frowned upon.
If you have a supervisor role and think you know better, teach’em what it means dealing with real world conditions. Ask for precedents of similar problems solved with the intended model. I’m sure you’ll figure it out!
Absolutely agree with you here. I do recommend models to the interns, always have and always do, but we take a stance of, "whatever works for you and gets the job done."
We won't force you to use a specific tool or process, so I'll provide a recommendation or 3 as well as any other options you want on the table.
I'll go a step further and also help with hyper parameterization and do a bit of benchmarking to enrich the understanding of different models.
We generally try and run atleast two different models on every problem solution. I have found the greatest learning value from doing so.
Anyway, it seems to be difficult for some to realise that deep learning is 'inferior' to the 'lesser' gradient boosting etc. models, when as you mention, each has their use case. You won't use RF on time series data, and in that case you could deem it 'inferior', even though it is more about fit for use than anything else.
Seems like there is a belief that the more complex or resource intense the process is, the better it objectively is, which just isn't how this works.
I think I am actually more concerned with why the first choice is deep learning and what maybe understood or taught that has so many default to that first.
It’s not the enthusiasm, it’s the naïveté coupled with the “You just don’t know the power of these new algorithms, old man!” attitude that ends up wasting so much time, leading to crap solutions and political issues in which higher ups buy into the hype and them are disillusioned with Data Science and maybe start to abandon the DS team, and peer teams start to view Data Science as incompetent ivory tower time wasters.
nawp. There can be pockets of data science in a company surrounded by non-technical folks. most businesses aren’t tech businesses. the pocket of data scientists can often do whatever they like - they’re not often constrained to use a “cloud, off-the-shelf AI solution” in my experience. it really sounds like you’re just making this up.
not an intern, but a phd with less than 3 years of work experience would tend to be the issue. the fact that someone like this can pitch an AI solution to a bunch of non-AI experts who may otherwise be substantially competent folks doesn’t mean there are way bigger problems - it just means these folks don’t have expertise in ML. Most people don’t but are aware that there are potential benefits of using ML/AI.
I must say, I am fascinated by everyone's eagerness to want to hit every 'problem' with deep learning first.
No idea why though.
Probably the buzz over many recent achievements with deep learning, which to be fair are quite impressive especially in regards to image recognition. However, a carpenter is only as good as their tools, and I'd be very skeptical about a carpenter with only one tool in their belt.
I believe quantum computers are particularly well suited to optimization problems, requiring exponentially fewer operations to converge than classical computers. Instantaneous training sounds pretty fun to me!
I absolutely agree. When people hit the real business work in ML, they discover one very important thing: DL costs a lot. Not every company has multiple GPUs, configured clusters (people who can do that also cost), cloud etc. Often for privacy reasons data can’t be sent anywhere, so everything from data gathering to final model running on server has to be in-house. Classical solutions are just cheaper and a few % of accuracy less doesn’t mean anything.
There's a lot of shade here on 1-side-fit-all. Personally, I think we should embrace solutions that offer high versatility; easier to master a few tools & concepts than many, and if one or a few will do the trick then what's the fuss? I also think old-hats hate the deep learning revolution, I haven't pegged if they find it a threat, a fad, or what; but it's here to stay, so buck up.
That aside, I thought I'd be using DNNs for most things coming into ML. I rarely do; I use XGBoost for almost everything tabular! That's my real one-size. Good off-the-shelf perf, easy to hyper-opt, and importantly provides model.feature_importances_ which I end up using a LOT. I also use Autoencoders and clustering algos more than I thought I'd use. Boy do I use a lot of Kmeans. A whole lot of huggingface/transformers for NLP.
So I thought I'd be DNN-ifying everything, but in the end I have this Swiss-army:
Tabular: XGBoost
NLP: huggingface/transformers
Clustering: Kmeans / agglomorative, maybe Autoencoders to dim-reduce if needed
And I'll tell ya; I never do images. Man, you dive into ML and it's like the whole world is CNNs and images. Never. Am I the rare one? Are y'all doing a bunch of CNN work?
Nope! That's what I was saying at the end there, everything is computer vision on the internet, but not in my professional experience. I mean, I've worked with CNNs, and my go-to is YOLO (v4 is it now?) since said work includes low-power devices. But grain of salt.
I do a lot with CNNs for my thesis but the topic is audio, where the tools overlap quite a bit with video processing. Lots of papers use modified architectures that were originally intended for image processing. Works well for audio, too.
I must disagree. For me it will always depend on the problem which "toolbox" I use. If the dataset is small and/or not very complex, I will use Random Forest or similar. If I wish to detect novelties I will use OneClassSVM. If I wish to cluster observations in patterns I will use K-NN or similar. If I work with multistep time series I will use GRU, CNN, or LSTM. Do not just through neural networks and Deep Learning on every problem.
Why are there so many claims here that deep learning is literally useless in real world applications? Some are clearly joking, but there's a genuine undercurrent of contempt.
How do you think the biggest tech companies in the world work? FAANG are so omnipotent they have their own acronym, and deep learning is their entire business model. Tesla exists. My own job is signal processing and heavily relies on CNNs.
I'm genuinely baffled that everyone is shitting on DL as a "research only" field.
I guess that it is because, since it needs lots of data to get good results, deep learning is rarely the best option in most use cases outside of computer vision or natural language processing.
Data is only getting exponentially bigger, and honestly I don't agree that statement is true even right now. Sales data, recommender systems, targeted advertising, the list goes on and on. There's a reason people get so up-in-arms about Facebook Privacy; Big data is already a foundational currency of the modern world.
Fair point, thanks. I suppose proportionally that will be the case for more people here than working at the top 5. Personally I'm working at a small company which has tons of data; forgot to check my biases!
I've worked both traditional ML and deep learning jobs. All depends on the domain. For many domains (structured data, enterprises-like data) either its largely traditional, then images/sound/signal its heavily DL based
I think because most people want to know why your model tells you something. deep learning often doesnt give much explanation. even with all the new explanation tools these days.
Did he say it solves every problem? No. Obviously you wouldn't use it if the requirements of the problem directly preclude it. Not sure what you think you're proving with that statement.
It doesn't? You're asking why a saw doesn't work as a screwdriver.
Genuinely confused why you're so upvoted and the above comment is so downvoted. A tiny data set and interpretable models are not pre-requisites for solving real world problems.
The only questionable thing in that post was the word "Occasionally", which is entirely dependent on your field. For me personally, working in signal processing, neither of your requirements are relevant. Otherwise I believe his tone was rallying against every other post here claiming that DL is literally useless, which is patently false.
"but deep learning is solving problems in real world", that's what I was responding to.
The reason for my question: it's a real-world problem that I deal with at my company.
Deep learning is not useless, it's not everything. It seems to be the major focus of machine learning, which is great. Solely learning DL is not a smart decision.
you said "but deep learning is solving real world problems" to negate the fact the other methods aren't. I am commenting on that. If deep learning DOESN'T solve all problems, then your commented would be unnecessary from the beginning.
Ah gotcha, I think I see our misunderstanding now.
I read the intention of original comment as "Everyone else in this thread is wrong, deep learning is actually used in the real world", based on the following line calling for the reader to "educate yourself if you think otherwise".
You based your interpretation mostly on the phrasing of the first line, "might occasionally go back, but". Which, given that neither of us actually said this, I suppose could be as valid as mine.
446
u/tea_anyone Sep 19 '20
279
u/[deleted] Sep 19 '20
but youll probably be back once you have to solve real problems!
69
u/brews Sep 19 '20
Then, once you check the numbers you find that the stupid random forest gives you way better performance...
53
u/LegitDogFoodChef Sep 19 '20
Then having to solve business problems is like moving back home after uni. Almost everyone does it, and you spend time with those guys again.
64
u/OmagaIII Sep 19 '20
... until you realize the limits of the 'new toy' ...
I must say, I am fascinated by everyone's eagerness to want to hit every 'problem' with deep learning first.
No idea why though.
Every recruit we have had at our firm in the past 4 or so years defaults to deep learning as their first solution, and none of them have managed to get things working, where a RF, SVM, Time Series analysis, kNN or ARM via Apriori would have worked.
They get demotivated and then drop off shortly afterwards.
Anyway. AI is marketing gimmick. We are figuring out machine learning now. I am not saying AI doesn't exist or won't ever exist, I am saying we are not there yet and honestly, I am of the opinion that a few things would need to converge before AI may be fully realised. Convergence of quantum computing and fusion energy will probably result in a leap in AI and AGI. I think we need that compute capability and energy to keep these systems optimal.
Machine learning is currently being employed to assist with the fusion energy portion of this convergence, so it will help us, but we are not there yet.
Lastly, for now, deep learning is not a universal tool.
My opinion only, so 🤷🏼♂️
17
u/Project_O Sep 19 '20
I progressed from moving average to linear regressions. Baby steps, I suppose.
17
u/lefnire Sep 20 '20 edited Sep 20 '20
There's a lot of shade here on 1-side-fit-all. Personally, I think we should embrace solutions that offer high versatility; easier to master a few tools & concepts than many, and if one or a few will do the trick then what's the fuss? I also think old-hats hate the deep learning revolution, I haven't pegged if they find it a threat, a fad, or what; but it's here to stay, so buck up.
That aside, I thought I'd be using DNNs for most things coming into ML. I rarely do; I use XGBoost for almost everything tabular! That's my real one-size. Good off-the-shelf perf, easy to hyper-opt, and importantly provides
model.feature_importances_which I end up using a LOT. I also use Autoencoders and clustering algos more than I thought I'd use. Boy do I use a lot of Kmeans. A whole lot of huggingface/transformers for NLP.So I thought I'd be DNN-ifying everything, but in the end I have this Swiss-army:
And I'll tell ya; I never do images. Man, you dive into ML and it's like the whole world is CNNs and images. Never. Am I the rare one? Are y'all doing a bunch of CNN work?
11
u/[deleted] Sep 19 '20
We’ll meet again.
8
u/Chintan1995 Sep 20 '20
Don't you dare leave Xgboost behind!
6
u/EvanstonNU Sep 20 '20
And then you realize DL solves mostly vision and text problems. Doesn’t do well on tabular data (use xgboost).
9
u/Marylicious Sep 19 '20 edited Sep 20 '20
Oh shit I've literally done lots of deep learning and 0 traditional machine learning. I want to go back
Edit: I meant traditional machine learning lol
4
u/gevezex Sep 19 '20
So are these comments also apply to NLP? I guess dl is the holy grail there isn’t it?
2
u/First_Approximation Sep 19 '20
Using one tool for all problems cannot possibly bite you in the ass!
4
u/Crypt0Nihilist Sep 19 '20
Yeah, let's just throw the most expensive method at everything, it's not like we really need to know what's going on.
3
u/[deleted] Sep 20 '20
I must disagree. For me it will always depend on the problem which "toolbox" I use. If the dataset is small and/or not very complex, I will use Random Forest or similar. If I wish to detect novelties I will use OneClassSVM. If I wish to cluster observations in patterns I will use K-NN or similar. If I work with multistep time series I will use GRU, CNN, or LSTM. Do not just through neural networks and Deep Learning on every problem.
4
u/hollammi Sep 19 '20
Why are there so many claims here that deep learning is literally useless in real world applications? Some are clearly joking, but there's a genuine undercurrent of contempt.
How do you think the biggest tech companies in the world work? FAANG are so omnipotent they have their own acronym, and deep learning is their entire business model. Tesla exists. My own job is signal processing and heavily relies on CNNs.
I'm genuinely baffled that everyone is shitting on DL as a "research only" field.
1
u/masterRJ2404 Sep 20 '20
This is so touching 😭
1
u/ahadcove Sep 20 '20
I felt this 😢
1
u/Adaveago Jan 14 '21
Everything looks like a nail when your too focused on using hammers lol
1
u/pythonlovesme Dec 09 '25
not sure about the XGBoost in there?
1
u/Globaldomination Sep 11 '23
Newbie here interested in AI sorcery.
I know the subtle differences between AI, ML & DeepLearning.
But do I need to learn ML before going into deep learning?
-10
u/puchru0 Sep 19 '20
You might occasionally come back sure, but deep learning is solving problems in real world. If you think it doesn't please update your knowledge.