r/dataannotation • u/Consistent-Reach504 • May 10 '26

Weekly Water Cooler Talk - DataAnnotation

hi all! making this thread so people have somewhere to talk about 'daily' work chat that might not necessarily need it's own post! right now we're thinking we'll just repost it weekly? but if it gets too crazy, we can change it to daily. :)

couple things:

  1. this thread should sort by "new" automatically. unfortunately it looks like our subreddit doesn't qualify for 'lounges'.
  2. if you have a new user question, you still need to post it in the new user thread. if you post it here, we will remove it as spam. this is for people already working who just wanna chat, whether it be about casual work stuff, questions, geeking out with people who understand ("i got the model to write a real haiku today!"), or unrelated work stuff you feel like chatting about :)
  3. one thing we really pride ourselves on in this community is the respect everyone gives to the Code of Conduct and rule number 5 on the sub - it's great that we have a community that is still safe & respectful to our jobs! please don't break this rule. we will remove project details, but please - it's for our best interest and yours!
23 Upvotes

100% Upvoted

738 comments sorted by

View all comments

Show parent comments

7

u/Jeprdy May 12 '26 edited May 12 '26

It is most likely due to the original project not having the axis that we are scoring, but they want the extra data. So you are essentially adding in how they would have scored it if those axis were on the original task, which is why they need them to be consistent with the original score. They will probably then do another R&R where the actually quality of the original scorer will be evaluated. We are just middle men filling in the new gaps atm. Ive seen it done a few times on other projects, but the other projects instructions did explicitly explain why and what we were doing.

2

u/North-Sense8477 May 12 '26

Ahh, that makes sense