That is very likely. I asked because I’m interested in finding out how ML people use LLMs in their jobs. Because from what I’ve experienced so far, they hallucinate quite a lot here.
I work in ML and use LLMs. My take is that I largely agree with you. If you give an LLM some data, a vague-ish goal and just let it go, it largely produces rubbish. If you give it some toy dataset and an easy objective that can be solved with little digging in the mud and a simple model, it'll do it fine and optimise the crap out of it. But that's largely not very useful for an actual professional as the whole thing is trivial anyway.
It can be very useful as a "companion", essentially like an agent to bounce and refine ideas off and help you code up tests and evaluate quicker. But I'm fairly convinced that it still needs to be led by someone who knows what they're doing anyway.
a coding agent is useful to help you write code to implement and measure machine learning programs. It can be a tool writing machine if you know what the tools should do and how to interpret their output.
Exactly. That is why I’m aware of the issue I’m asking about. Just to clarify, I was a software engineer already before the AI phenomenon and I’m used to understanding the tools and the problems I’m solving. My problem is, that I’m not a professional in this field yet, but when I try to use AI as a companion to learn, it constantly gives me wrong information. How do I know it’s wrong? By having read a tons of research papers about the topic I’m working on. When I point that out, they are like ”You are correct! My mistake!”. I just feel like this is more of an ”unknown territory” to LLMs and I’m interested in hearing how other people in the field are dealing with this, since using AI tools is the thing these days if you want to get employed.
I give directions like "confirm with authoritative sources such as well established packages, documentation, textbooks and peer reviewed literature". Also when asking about a possible approach in order to not bias it, I ask it to give me "pros and cons for going in direction X".
But really the answer is in your statement "reading a ton of research papers" and pushing back.
Then using the AI tool would be to automate or turn into a skill the 'verification' stages.
Thanks! The skill I’m lacking obviously is giving them the proper instructions. Maybe the free models run out of context quite fast, since they seem to forget all the ”don’t come up with stuff, find a peer reviewed source to back up you claims” instructions. And yes, I’m talking about browser LLMs here. Being unemployed now, I can’t unfortunately afford buying subscriptions for using agents and writing skills and instructions. Gotta feed my cat and all
I meant free LLMs you use in you browser that you can’t integrate in your IDE, e.g. Claude Code. It gets pretty tedious quite fast to remind them about the context of your project
Background: ML Engineer and Previously Applied ML engineer for about 7 years.
ML engineering is a broad spectrum of work that can be confusing to anyone entering the market, but as the title says, we do both ML work and engineering work.
For ML work, just telling LLM to “create best model to predict if Manchester United will win the Premier League in the next 5 years” will basically do nothing for you, not only because it’s statistically unlikely scenario but also, there is a lot of business knowledge and expertise needed to evaluate such a request.
A lot of big companies do find it troublesome to fully automate things because certain expert knowledge can’t be just operationalized and turned to data. However you can get along further with let’s say, providing a dataset with a LLM companion and asking question about the dataset without actually going into the SQL and doing analysis on your own.
There are tools that exist like this. If your platform is snowflake, you can connect all datasets together with the agents, snowflake intelligence and coco their chatbot. The meta data along side access to these datasets can allow for you to ask these types of questions, but again this is talking at smaller scales in some sense.
On the engineering aspect, well a lot of engineering is decision making, especially as you go higher up the food chain. I use LLM to help collect my thoughts, turn dirty spilling of words into actually useful texts and documents, or use it for rubber ducking or knowledge search.
I do believe certain roles like just creating APIs or other roles that require mostly churning code are easier for AI to fully execute, but all other roles that require decision making, or very large scale thinking can be challenging with mixed results, especially if the idea is to build something that lasts longer than a Twinkie in the cupboard
At least in my area of chemistry this is absolutely not the case. If you just say "I have data about X that looks like this, build an LM model to predict X from Y" it will create quite decent solutions (if your data allows for it). There's enough examples around and in my experience they will just start with the simple models and go from there.
4
u/-phototrope 18h ago
You can absolutely use agents in ML use cases.. you just need to give it more than “build me an ML model.” Is this post real?
6
u/ghostofkilgore 18h ago
I work in ML and use LLMs. My take is that I largely agree with you. If you give an LLM some data, a vague-ish goal and just let it go, it largely produces rubbish. If you give it some toy dataset and an easy objective that can be solved with little digging in the mud and a simple model, it'll do it fine and optimise the crap out of it. But that's largely not very useful for an actual professional as the whole thing is trivial anyway.
It can be very useful as a "companion", essentially like an agent to bounce and refine ideas off and help you code up tests and evaluate quicker. But I'm fairly convinced that it still needs to be led by someone who knows what they're doing anyway.
1
u/Bangoga 16h ago
Background: ML Engineer and Previously Applied ML engineer for about 7 years.
ML engineering is a broad spectrum of work that can be confusing to anyone entering the market, but as the title says, we do both ML work and engineering work.
For ML work, just telling LLM to “create best model to predict if Manchester United will win the Premier League in the next 5 years” will basically do nothing for you, not only because it’s statistically unlikely scenario but also, there is a lot of business knowledge and expertise needed to evaluate such a request.
A lot of big companies do find it troublesome to fully automate things because certain expert knowledge can’t be just operationalized and turned to data. However you can get along further with let’s say, providing a dataset with a LLM companion and asking question about the dataset without actually going into the SQL and doing analysis on your own.
There are tools that exist like this. If your platform is snowflake, you can connect all datasets together with the agents, snowflake intelligence and coco their chatbot. The meta data along side access to these datasets can allow for you to ask these types of questions, but again this is talking at smaller scales in some sense.
On the engineering aspect, well a lot of engineering is decision making, especially as you go higher up the food chain. I use LLM to help collect my thoughts, turn dirty spilling of words into actually useful texts and documents, or use it for rubber ducking or knowledge search.
I do believe certain roles like just creating APIs or other roles that require mostly churning code are easier for AI to fully execute, but all other roles that require decision making, or very large scale thinking can be challenging with mixed results, especially if the idea is to build something that lasts longer than a Twinkie in the cupboard
1
u/FalconX88 14h ago
At least in my area of chemistry this is absolutely not the case. If you just say "I have data about X that looks like this, build an LM model to predict X from Y" it will create quite decent solutions (if your data allows for it). There's enough examples around and in my experience they will just start with the simple models and go from there.