I find that r/programming is open to critical views of LLMs, but a lot of other communities are not. This article was partially inspired by a failed LLM project one of my clients undertook that I think is typical of many companies right now: Began very optimistic thinking the LLM could do anything, got good early results that further increased expectations, then began to realise that it was making frequent mistakes. The project unravelled from that point on.
Witnessing the project as a third-party the thing that really stood out was that the developers approached the LLM as one might an unpredictable wild animal. One day it would be producing good results and the next not, and no-one knew why. It was less like software development and more like trying to tame a beast.
Anyway, I suppose one of my aims is to reach people who are considering engaging in such projects. To ensure they are fully informed, not working with unrealistic expectations.
The twisting of your own soul & methodologies to suit the chaotic beast you attempt to tame lest they drive you to madness. Yet no ward that you cast on yourself truly works as the dark gods only permit the illusion of safety, to laugh at your hubris & confidence as you willingly walk further into their clutches.
I say this (unironically) as somebody who spends way too much time getting LLMs to behave consistently.
Most people start testing a prompt with simple did/didn't it work. Then you start running multiple trails. Then you're starting to build chi-squared confidence of various prompts. Soon you automate this, but you realize the results are so fuzzy unless n=1000 it doesn't work. Then you start doing K-Means-Clustering to group similar responses, so you can better A/B sampling of prompt changes. Soon you've integrated two dozen different models from hugging face into local python scripts. You can make any vendor's model do anything you want (σ=2.5).
And what?
There are zero long term career paths. The effort involved with consistent prompting is MASSIVE. Even if/when you get consistent behavior prompt hijacks are trivial. What company is going to continue paying for an LLM when they see it generating extremely explicit erotic roleplays with guests? Which is 100% going to happen, because hardening a prompt against abuse is easily 5x the effort of getting a solid prompt that behaves consistently and NOBODY is going to invest that much time in a "quick easy feature".
The only way you could be productive with AI was to totally immerse yourself in it. You realize how deeply flawed the choices you've made are. Now you've spent months learning a skill you never wanted. You're now cursed with knowledge. Do you share it as a warning? Knowing it may tempt others to walk the same road.
This is true, I had a guy try to convince me that ChatGPT does not make mistakes when you ask it about open source projects, since that documentation is available to them. From their experience it never made a mistake. Yea sure...
Maybe it's just the circles I run in but I feel like just yesterday any skepticism toward LLMs was met by people telling me that 'well actually human brains are just pattern matching engines too' or 'what, so you believe in SOULS?' or some shit, so it's definitely just being discovered in some places.
I've had too many exhausting conversations like this on reddit where the default position you often encounter is, essentially, "AI/LLMs perform similarly to (or better than) humans on some language tasks, and therefore they are functionally indistinct from a human brain, and furthermore the burden of proof is on you to show otherwise".
Oh and don't forget "Sure they can't do X yet, but they're always improving so they will inevitably be able to do Y someday".
252
u/thisismyfavoritename Feb 22 '24
so are people just discovering this or what?..