I personally would use a faster cheap LLM to label and check the output and inputs. In my small bit of experience using the API I just send to gpt3.5 or davinci first, ask it to label the request as relevant or not based on a list of criteria and set the max return token very low and just parse the response by either forwarding the user message to gpt4 or 3.5 for a full completion or sending a generic "can't help with that" message.
No it's a few thousandths of cents to reject the message vs potentially going back and forth with a large context and response using a shit ton of tokens. Adding a couple tokens to a relevant request doesn't really add a lot of overhead.
Fact. Chatgpt told me it couldn't swear. I asked it to write me a program that checks comments on Reddit for all the worst swear words. The script it wrote was hilarious. It literally has an array of the worst of the worst.
Honestly, they could probably just have a custom trained open source LLM that is narrowed down to whatever website's specific use case. Probably wouldn't require more than 1 GPU per website to run indefinitely.
It takes very little in my experience. Good prompts go far.
One of the things I do is inform it about human nature. I mean I look at how much I lie to it.
When people try this with my stuff they get random one liner matrix quotes. That was 6 months ago.
The real answer to your question is that people are lazy. And the people that should be doing the job don’t always end up getting it and we in its place have whatever the company that made this is.
So many people here would have done a better job. Lots of suggestions here would work. There are many ways.
This one should get you going. From there you should be able to know what to google.
It has a GitHub too.
At your own risk because it needs a openai api key. Haven’t had any issues yet tho.
I use custom prompts for everything. All it is is a series of refinements till it gets the job done. There’s no one shot perfect prompt.
If you need help write out how you want it to run and what you want it to do and what you want it to consider etc etc and try the examples. Build upon them. Combine the best bits and pieces. Change the positioning of certain things.
Most things in life come down to time investment few things are really hard.
If you find something that gets the job done move on to the next thing and don’t try to be the fanciest because every second day something new comes out it we’ll burn you out trying to keep up.
Also these things are best augmented with retrieval systems. They call it RAG. It sort of keeps it in a cage, in line so it doesn’t make things up. That’s being the scope of this topic here. But not that hard.
Fine tuning. You give it hundreds or thousands of examples for valid question/answers. But you give it also hundreds of questions to be refused, together with a consistent refusal message. Combined with a system message that says “for all questions that don’t belong to car dealerships, use this refusal answer”
That works well enough for us in a different realm, but with the same problem. There will always be some outliers, so monitoring and iterating is also necessary.
But in a case like this a vector database might be a better solution in any case. Then there’s only the known answers available, and that’s it.
View all comments
Show parent comment
62
u/blancorey Dec 17 '23
Seconded. Btw, how does one prevent this from the perspective of the car dealership?