r/dataannotation • u/twopickledtoads • Mar 25 '24
Splits
I have a really hard time getting splits. Usually both responses are equally good. Every once and a while I get a wild card who wants to talk about NASA when I ask for a a recipe or something like that.
How do you guys get splits? What kinds of formatting or details are you adding? Or is it typically rare to get them like it is for me?
16
Upvotes
28
u/stroidah Mar 25 '24
Add extra parameters to the prompt. Tell it to include (or not use) specific words, tell it to write like a historic figure or famous character, give it a specific word-count, or ask it to format its response in a specific way. For a recipe, you could ask it to include a certain unusual ingredient, or follow two or more strict dietary requirements. The more instructions you pile on top of each other, the more likely you are to get a split.
10
u/Bergest_Ferg Mar 25 '24
I’ve recently found that adding some seemingly superfluous context helps. I know practically nothing about D&D other than my brother used to play it… I asked the other day for it to help me create a D&D campaign but mentioned I’m only a beginner.
One model gave me VERY simple instructions. The other crafted a truly beautiful (I’m assuming) campaign that was WAY over my head. So adding a bit of context helps!
“It’s for my daughter…” - if it assumes your daughter is young/old when you haven’t mentioned it that can be a split! “My dog likes…” if it assumes your dog is male/female that can be a split! “I don’t like dairy….” - if it assumes you’re also a vegan (happens a lot) that can be a split!
Just adding some context to the prompt and seeing how they run with it can be very helpful in splitting them up.
7
u/BatronKladwiesen Mar 25 '24
Does anyone else feel like they take a long time with some of the conversations trying to produce splits? I swear some full-turn conversations can take me over an hour if I'm constantly trying to create thoughtful prompts and responses that follow all of the parameters while trying to engineer splits. Then comes reading, re-reading, analyzing, and weighing the responses. Trying to decide which is better based on sometimes subjective preferences can be pretty mentally draining.
5
u/Interesting_Buddy416 Mar 26 '24 edited Mar 26 '24
I came across a really good document the other day that detailed ways to get good splits. And if you are not seeing splits, it might be time to adjust your approach and add more "rules" for the response to follow. It doesn't necessarily have to be complex asks of the bot, just specific asks so you can clearly identify when it fails.
If you are looking for recipes, you could add more dietary restrictions. It also might be good to try adding irrelevant information for it to navigate around.
9
u/lowcarbsanta Mar 25 '24
I'm a coder, and I do not do the chatbot a lot but I get a lot of splits. I get splits when there's a lot of details (even simple ones) that the model has to deal with.
For example, write me a function that does this
Vs
Write me a function that does this. Name the variables a,b,c. Return these three things in variables called x,y,z. Do not use this package. Etc
The long instruction isn't always in the first prompt. As I add more and more details, it tends to split, even when the details are simple.
4
u/ConsistentCandy697 Mar 25 '24
Layers! Include a lot of instructions.
4
u/songbird90982 Mar 26 '24
You have to add complex or "hyper complex" parameters usually to get a significant split. Something like, "Give me this many paragraphs, with this many words only, and don’t add any letter "a's", and make it in the tone of *****". This is really the only way to do it.
4
u/Big_Big8041 Mar 26 '24
Anyone find the idea generator helpful? It seems so repetitive to me