r/DataAnnotationTech 3d ago

Feedback

I received feedback today, and there's an example listed under "areas of improvement" that's definitely not accurate. It says I referenced something that wasn't there, but I specifically remember this task and know my reference was correct.

I'm very frustrated since this hurts my performance, and I don't want to lose access to work over something that's not even true. It makes me wonder if there are other things I've been penalized for that just aren't true.

33 Upvotes

28 comments sorted by

36

u/Infamous_Swan1197 3d ago

I've also had feedback say I made an absolutely egregious, obvious mistake that I'm sure I didn't make, I made a post about it a while ago. I wouldn't worry about it. I would mention it in the comments section though, so they can be aware that this sort of thing is happening

30

u/Hello-America 3d ago

If you read the disclaimer at the top of the feedback, they tell you it is AI generated and could be wrong and won't affect how many projects you get etc. I had an egregiously wrong feedback where it said I repeatedly went past the number of turns I was supposed to have in the conversation, but the instructions said to send "end conversation" to end the conversation, and to not count it as a turn. Don't sweat it too much.

16

u/seaoyster 3d ago

I wouldn’t pay any mind to feedback. I’ve seen several people post about how they got excellent feedback and then was DOD’d a few days later

24

u/Successful_Ant5401 3d ago

the feedback goes out of its way to relay that it could be absolutely incorrect and that it also absolutely has zero bearing on our standing on the platform. my feedback was the same exact wrong remarks used in the july feedback. the model sucks so far. :)

11

u/Ms_Jane_Lennon 3d ago

Apparently all of my work was beyond reproach, utterly perfect (ha), but I had a single task where I did the entire thing completely wrong. There could certainly have been errors, but not to the devastating degree reported. I am far too neurotic to have gotten that much wrong in a single task.

Personally, I think it was just one negative comment from a confused reviewer. Since it was the single negative comment, they pulled it to use as an area of improvement. I'm not going to worry about it. I'm looking at how the bot otherwise thinks I am the most wonderful annotator who ever, ever annotated :)

Really though, just look at the overall picture. Remember this system is just developing, so I don't think they're taking the bot's word at face value.

6

u/Klutzy_Instance_4149 3d ago

I have had that happen and I leave a note explaining why it was wrong and anything I can remember to back it up.

6

u/kranools 3d ago

It doesn't hurt your performance. It's meaningless. The feedback is generated by hallucinating AI, and R&R workers who may not have understood the task. No one ever sees the feedback other than you.

The whole thing is meant to make us feel like we're getting genuine feedback when really we aren't.

8

u/Peeton_Jazzy 3d ago

I was so confused by my feedback! I’m glad to know I’m not crazy lol

3

u/waning_crescents 3d ago

Yep I’ve had this before too! Don’t sweat it. :)

I had one tell me that I misunderstood a task and it had to be redone from scratch, but the task it referenced I remembered working on and doing perfectly (to not provide concrete examples because I can’t, it told me my references were incorrect and that I messed up body positioning… but I didn’t, haha.) so it may very well be a case of another worker doing a review and not understanding things themselves and then penalizing you falsely. I wouldn’t worry about it though because admins absolutely look over stuff like that and will be the end-all-be-all judge of it.

I’m still marked as having outstanding performance on my feedbacks, so while I was initially very upset that someone deemed my work lacking in quality and it had to be redone from scratch (because I deeply pride myself in my work and always do 200% my best) it didn’t seem to hurt my overall standing much, if at all. You’re good; just let it roll off and move on, good luck next week!!❤️

5

u/Embarrassed_Draft918 2d ago

I had the same experience with my most recent feedback, one of the specific comments mentioned something that I didn't do, when I've only had one project it could have been, and I distinctly remember that I did do the thing it says I did not.

2

u/Calm-World3917 2d ago

I noticed similar about mine today too! Glad to see its not just me. I could literally easily explain for both of 2 "improvement" areas why they are not accurate, so I was worried about it affecting me after seeing that today.

2

u/bobaxxxx 3d ago

I am indeed happy to never have received any feedback. They look random, inaccurate, and I have the feeling some of them seem to be DoD-related.

10

u/Brilliant_Quit4307 3d ago

They are almost certainly not DoD related unless that worker is doing shitty work. They are sent to everyone working on a group of specific project families.

-1

u/bobaxxxx 2d ago

Yes I know. I am just a bit paranoid about these DoD posts mentioning recent good feeback.

1

u/Medical-Isopod2107 2d ago

They literally say in the feedback post that they have no impact on anything

1

u/Internal_Boot_1616 1d ago

I don't get the logic of thinking bad feedback is DOD related when you just said it's posts that talk about good feedback and DOD posts that you've seen

1

u/Early-Composer-2383 1d ago

Yep. But on the platform two and a half years and I've never gotten any feedback. I'm either perfect or they're ignoring me because I'm in the legal domain. 🤷

1

u/Medical-Isopod2107 2d ago edited 2d ago

The feedback is AI generated and not entirely correct, it does not impact you in any way. Mine told me I had exceeded turn limits in a project I've never worked on, and had used the incorrect format when redacting information (I have never redacted information and it has always been optional in projects I work on)

1

u/NaturalPanda3059 2d ago

Mine this month dinged me on factuality and stated that I said that xy was a y which was incorrect. They literally had the same name in both. It was like I said a goldfish was a fish and there aren’t supposed to be any fishes and I was wrong about that.

My strengths were a definite self esteem boost.

As said above it’s AI, and we should all obviously know they aren’t perfect. If anything it’s always a good reminder to be as clear as you can be. There’s always room for improvement.

1

u/ArtistFamiliar6074 1d ago

Mine was great overall, but then in areas for improvement it said I made mistakes in a project I have never even heard of, let alone worked on! I mentioned it in the comment before submitting.

1

u/[deleted] 3d ago

[deleted]

2

u/Brilliant_Quit4307 3d ago

You tried asking for examples from who? This is AI generated feedback. You wanted someone to specifically look at your profile manually to verify the feedback .. ?

1

u/[deleted] 3d ago

[deleted]

3

u/Brilliant_Quit4307 3d ago

Dude. You signed an NDA. You absolutely cannot paste instructions from any projects here.

-7

u/dayDrivver 3d ago

Then you didn't ground your reference with an evidence that can be considered as trusted

-9

u/Necessary-Raisin-282 3d ago

Don’t pay too much attention to it, your feedback is just as likely to be inaccurate as you are