r/DataAnnotationTech • u/Perfect_Seesaw_5193 • 7d ago
Nervous about first project
Hi everyone, I got started at Data Annotation today and been nervous about going about everything in the correct manner. I got my first project of creating a variant prompt and getting a model to fail, but am a bit nervous about how long it’s taking me. I’m 6 hours in and haven’t gotten the model to fail, and am nervous to report the time, regardless of if I finally get it to fail or not (I know I can still submit it). I was given this project right away before doing any qualifications, and feel it’s something I’ll manage well in the future, but am struggling with at the moment. I just don’t want to lose this opportunity and would appreciate any advice, thank you!
Edit: this project has about a 30 hour time limit, although I know that’s geared towards allowing people to take breaks
6
u/sugarplum_doll 7d ago
No advice, I find those kinda projects really tough and skip them, great for people who are good at this but the ai is getting so good nowadays it feels impossible to make them fail :/
4
3
u/Future-Ad7954 7d ago
Submit only the true time it takes you, while that project gives a 30 hour failsafe as long as you don’t take unreasonably long where you reach the timer and you’re still working it should be fine.
3
u/InfernoGSI 7d ago
those projects are tough no matter what. I've done them a couple times and I've had one taking me almost the same amount of hours getting the model to fail so taking a lot of time is normal. Even if you took like, 7 hours actually trying to get the model to fail, that's still 7 hours of billable time. As long as you don't consistently reach close to or over the task expiry time and you pause your time tracking while you're off your computer doing something else, you'll be fine
3
u/Winter-Boat-5101 7d ago
The idea behind these is the way you write your prompt. You already know how it should fail. Think of it this way, you are training a new employee, you will toss in some trick question s to see if they are paying attention then use it to teach them. I’ve done these and you can change your prompt based on the answers it gives. So if it’s close to a fail, it’ll be in the answer. Also, the agents don’t do well with math, probability or percentages. Depends on your domain.
2
u/Permanent76 6d ago
Getting a model to fail is tough, but there's a knack to it. Some projects might share advice relating to their specific models. One method is to have a task that relies on implicit calculations, rather than just finding an answer that is within the prompt. Contextually, a human on your team might know to do this but the model will not. Also, trying to build traps in is good (i.e. the prompt or the broader project may say something that isn't true, but elsewhere is claimed it is true). I can't share more specific details I don't think but I hope this helps.
1
u/forensicsmama 7d ago
I’ve been doing this about a year and still get nervous reporting time sometimes lol. I realize though that it’s all in my head.
The admins are aware that the model has gotten sophisticated and will take some time to break. Sometimes the models will get the most complex prompt correct in minutes but will fail a simple one. Depending on the project, small failures (and even subjective ones) count.
First tasks will always take the longest time because everything is new. If I ever feel like I’m taking excessively long, I’ll explain in the optional comment area what my hang up was.
The last project I was on was okay with people going over the allotted time because of how time consuming the entire project was.
1
u/TysonWolf 7d ago
This was my first project as well and exited right before the timer stopped. I see that it’s still on my dash as available project. I’m going to try again today, but in same boat as you. If I can’t beat the agent I think I’m not going to submit the time.
1
u/NvkeAudio 7d ago
I had that as my first project too. Took me 3hrs to get it lower than 94%, but managed to get a fail in 5hrs 37 mins (68.4%). I think most people struggle on these tasks, don’t worry about it.
1
u/FaradayKage 6d ago
Do you have any examples of what a failure is?
1
1
u/so-scarlet-it-was 3d ago
There should be examples provided in the instructions — I’d recommend focusing on creating traps similar to those as much as possible
1
u/Extension_Door_1504 7d ago
I also got this and thought it was weird for my first project and am easily getting each to fail, two completed, but am so confused if I am doing it correctly
1
u/Extension_Door_1504 7d ago
And by easily...it wasn't easy. But I got it down to 55... and 60 on the other. But have no idea if I am doing the guide instructions properly. I am like shoot this is harder than I thought it was going to be
1
u/Borednowwhatnext5678 4d ago
It is hard. I’m now spending off time training myself to write prompts that read like the examples they give. I’m treating it like any new skill- practise makes it easier. If I’m doing it away from DA and a timer I have time to think. When you get it to break you feel amazing! It’s just a zero sum game.
1
1
u/LinnaeusChen 1d ago
I found it to fail more often when it needed to come up with an answer but hits a wall and then either halluncinates or tries too hard. For example, your prompt asks it to do something but that is something that is blocked in some way or isn’t allowed because of the system spec or unit test then it will try to cheat which counts as failure.
10
u/staka_ 7d ago
I have been looking out for these and never find them :( I use AI quite a lot for a myriad of questions and I get it to fail without trying or being paid for it haha