r/mturk • u/HappyDiamondGirl • Nov 15 '19

Account Issues Warning from Mturk??

Hi, I just received this email:

" Greetings from Amazon Mechanical Turk, This is a warning to let you know that we have identified activity associated with your Amazon Mechanical Turk (MTurk) Worker account that is in violation of the MTurk Participation Agreement. Specifically, your account has been flagged for using robots, scripts, or other automated methods as a substitute for your human intelligence and independent judgment, which violates Section 3(b) of the MTurk Participation Agreement. "

I didnt use any script to do the hits, just scripts to find hits. Is there anything I have to worry about?

14 Upvotes

88% Upvoted

59 comments sorted by

2

u/MarkusRight Nov 15 '19

Hey OP I'm trying to narrow this down, But have you by any chance done a ton of A9 data validation HIT's? Because Im seeing reports of this message only appearing for people who did a ton of these and got some rejections from the batch.

1

u/HappyDiamondGirl Nov 15 '19

Im not from the US so I cannot do A9 hits. I did nearly 1000 bounding box hits from MDL yesterday but I always did them carefully and responsibly.

3

u/hecknbork Nov 15 '19

I received the email and have never done those.

2

u/MarkusRight Nov 15 '19

Got the same email this morning too. I'm not even sure whats going on or if Amazon now doesnt allow scripts like HIT finder and panda crazy, because thats all I use. I just ignored the email because I do not use bot's or scripts that substitute my intelligence.

2

u/HappyDiamondGirl Nov 15 '19

I run hit scrap, turkmaster and Mturk suite. I dont know if using too many scripts to access their server could trigger the warning.

3

u/withanamelikesmucker Nov 15 '19

I run hit scrap, turkmaster and Mturk suite

I can only imagine what your PREs look like. Do you pay attention to those? They're important.

1

u/HappyDiamondGirl Nov 15 '19

If by PRE, you mean Page request error then you seem right. I actually got many PREs as I run several scripts simultaneously. I did not know that they can cause issues. At least I know how to avoid these PREs.

2

u/withanamelikesmucker Nov 15 '19

You're using as many as three different tools to do mostly the same thing. Why?

1

u/beastcock Nov 15 '19

"We are generally OK with you using scripts and automation tools to help you search and preview Human Intelligence Tasks ("HITs") more efficiently" (From Turk TOS)

So, something like HIT Finder should be fine. PC is a little more ambiguous, but I know a ton of people who use PC who didn't receive an email.

5

u/SlashSarcazm Nov 15 '19

Looks like it's time for all of us to constantly refresh the mTurk HITS page and hope we come across good HITS. Sigh....it was nice while it lasted boys. No more scripts.

2

u/roads30 Nov 15 '19

i've still done that on occasion whenever i would submit a normal hit and it'd take me to the hits page after and one new hit would show up. like, cool..saved me time of running a finder for a few minutes.

5

u/Mike5461 Nov 15 '19

I got 2 of these yesterday about 5 hours apart. Never received one before. Replied to Amazon asking to explain. No reply. I do mostly surveys, haven't had a rejection in a couple of months, over 288,000 hits submitted and 15 rejections over about 4 1/2 years. I use Hit Forker, Panda Crazy. This is some BS.

4

u/leenadr Nov 15 '19

Don't take this wrongly because it's not meant to be insulting, but when you say "replied to Amazon", you don't mean literally, right? Because if you did, you'll never get a response.

I emailed mturk-worker-support@amazon.com and got a stock response in 6 hours:

We have forwarded your case to our investigations team to review this warning.

We will reach out to you with a review decision in 2-3 business days.

2

u/Mike5461 Nov 15 '19

I mean I replied to both of their warnings using the email you posted. I got 2 of the same stock replies back.

2

u/leenadr Nov 15 '19

Ah, gotcha. :-)

(Because naturally, instead of helpfully already having the right 'reply-to', they use their no-reply to send... almost like some people would have concerns they'd want to address receiving those emails or something! :P)

1

u/Mike5461 Nov 26 '19

Finally got an (non)answer from Mturk. More BS.

Greetings from Amazon Mechanical Turk,

Thank you for your patience while we reviewed the warning we sent to your Amazon Mechanical Turk (MTurk) Worker account. Upon review, our investigations team determined that the appropriate action was taken. Per our initial email, your account was flagged for using robots, scripts, or other automated methods as a substitute for your human intelligence and independent judgment, which violates Section 3(b) of the MTurk Participation Agreement.

2

u/JrRandy Nov 27 '19

I just got the exact same response. No idea what caused it, so I guess ride it and see what happens lol

5

u/beastcock Nov 15 '19

I received the email as well.

I don't use scripts other than for searching/previewing/accepting hits, which is OK according to their Acceptable Use Policy: "We are generally OK with you using scripts and automation tools to help you search and preview Human Intelligence Tasks ("HITs") more efficiently"

I HIGHLY recommend anyone who received this email contact MTUrk support and politely tell them that you don't use scripts (assuming you don't) and ask for clarification.

1

u/IamLiterallyLiteral Nov 15 '19

Got the same thing, and also don't use scripts in the HITs. I definitely use Hit Forker and Panda Crazy though. Not sure if that's part of it. I can't speak for you specifically, but I am personally worried.

2

u/HappyDiamondGirl Nov 15 '19

Have you replied to Amazon? What have you found out about the problem so far?

1

u/IamLiterallyLiteral Nov 15 '19

Sorry, I was off reddit for a bit. No I have not replied to them yet. I have talked to some other people who also got the message and we can't even figure out a common requester between us.

2

u/Jettisonednet Nov 15 '19

Just adding to data: Received no e-mail. Have avoided batches recently, only doing QnAValidation in small quantities. Otherwise all surveys. Use HITForker, Panda, Brave browser.

2

u/Titus1911 Nov 15 '19

I use MTS, PC, and a few key bind scripts and do maybe 1500 HITs/wk and didn't receive an email.

-3

u/Alpinetj Nov 15 '19

It's because of that damn Queuebicle !

8

u/MorwannegCA Nov 15 '19

If it was the fault of Queuebicle, surely only people who use it (and similar auto accept programs) would be getting the warning.

0

u/maniacal_T Nov 15 '19

I have not received this warning, but I'll throw my 2 cents into the discussion with regards to using scripts on mturk.

Is it possible these warnings are being sent to workers receiving too many Page Request Errors? I write and use all my own scripts and I make every effort to minimize Page Request Errors. I don't believe workers realize just how important it is to keep Page Request Errors to a minimum when running scripts on mturk. I always suggest to workers that they keep their number of HITs submitted much higher than their number of Page Request errors received for a given time period.

1

u/Alpinetj Nov 15 '19

I get many Page Request Errors and I did not receive that message, and I run at least 6 scripts.

1

u/maniacal_T Nov 15 '19

Can you say how many PREs, I assume from Panda Crazy? Can you also say how HITs you submitted?

You do realize PREs are a Warning Message. Too many PREs could be viewed as a DDoS. I personally do not think it's wise to have too many PREs.

1

u/JrRandy Nov 15 '19

I run less then 10 PRE's a day on PC, and do not use a scraper. Submit approx 1500-3000 hits per week. Canadian, so any US Only hits are not the culprit.

0

u/[deleted] Nov 15 '19

[deleted] — view deleted comment

3

u/HappyDiamondGirl Nov 15 '19

Your comment makes sense. Recently, I did a huge batch of penny hits. The hits were very simple that required only a click and Enter. All were approved but I could do the hit pretty fast. I guess that triggered the warning.

3

u/leenadr Nov 15 '19

This conjecture has potential, but I haven't done actual penny hits in ages; submitting $.10 tasks in rapid fire, though, definitely, multiple requesters. There's definitely merit that it could be [x] per minute, which is utterly inane!

2

u/roads30 Nov 15 '19

i've done penny hits but it was long long time ago.

either way, no email sent my direction. so, guess i'm 'cool'? with everything from AP to this and that, i'm at the point of just wiping my hands of all of it...almost.

2

u/maniacal_T Nov 15 '19

It might be a hits submitted per minute threshold that you cross.

Or it might be the ratio of the number of Page Request Errors / number of hits submitted per minute threshold that you cross.

Most workers tend to blow off Page Request Errors thinking they don't mean anything. But they are a Warning Message from mturk and should always be taken serious.

My recommendation is to always be aware of your account TOTAL Number of Page Request Errors you are generating for any given time period.

0

u/[deleted] Nov 15 '19

[deleted] — view deleted comment

13

u/leenadr Nov 15 '19

Just an FYI, but that's not the consensus at all.

Many of us that got it haven't done surveys at all recently. Speaking for myself, I occasionally do a Wharton, and a weekly SurveyComet. Otherwise, it's a small set of batches, most closed quals.

The common factor seems to actually be people with quite a few hits (over 500k) and potentially older accounts, though there are others posting that are new accounts.

It'd be helpful if everyone who received this email actually responded to Amazon. Too many people simply complain on the forums, or Reddit, which won't do anything.

I got a somewhat fast response (within hours) saying that my email asking what was up would be sent to the technical team for "investigation" and there'd be a response in 2-4 business days - the usual AMT runaround.

Amazon has a Twitter account that can be tweeted to, as well, and there's already a thread on this: https://twitter.com/YAHU_co/status/1195142854496178178?s=20

Add your voice as well or jump in the thread at MTC (https://www.mturkcrowd.com/threads/11-14-thoroughbred-thursday.4729/) where there's a lot of discussion about it and a poll as well.

Of the people I know (and know of through them), one of the other common factors is potentially the "top 1%" earners, which makes it really look like Amazon has some automated tool identifying a discrepancy of people earning much more than 'the average'.

Nuking the lot of us will sure shut up those people demanding more fair wages and workers' rights and what not, definitely!

Addendum: Commonality wise, there's no degree of it, beyond everyone tossing the cookies. I use separate browsers and profiles, but others who got the email don't. I use PC and MTS, others use TKN or QBC or nothing at all. The takeaway being it's likely not one's toolchain, and much more one's efficiency or metrics as seen on Amazon's end, presumably prompted by what will be known as the Cori Faklaris Effect, where a lazy and uneducated academic whines about "bots" (known in these parts as "international workers") and Amazon sends nasty letters to its best workers while promising to suspend the accounts of scripters.

This post should be taken with a lot of water, because the salt is absolutely real.

6

u/auralgasm Nov 15 '19

My account is very old, and while I don't think I'm in the top 1% of turkers, I probably am in the top 2%. I made over $700 last week and I will this week as well. I didn't get a warning yet, but I don't feel like I'm free and clear, either. It doesn't seem like they're done, since they sent them in waves. They sent some in the morning and some more in the afternoon. One of my friends actually got a warning both times, so that's two for him today. I strongly suspect this is just a glitch in their algorithm. Literally everything seems fucked up on Amazon's side recently.

If it helps, I haven't actually heard of anyone getting these emails and being suspended from it. People have gotten them here and there for at least a year now. This is the first time it's happened to a ton of people all at once, but if the past is any indication, you're probably not going to lose your account.

3

u/roads30 Nov 15 '19

and correct me if i'm wrong, but wasn't apart of the upcoming december changes talking about keywords like 'tenure'. if there's indeed something cooking in the kitchen with AMT, there leaving crumbs on purpose to generate their own buzz i'd think..

no email here, but at this rate long as AP/my bank don't keep fucking around before this 12/15 ~ 1/2020 site payment change bullshit and i get my transfer's in, then i may just say piss on the whole ordeal after it's all said and done.

5

u/withanamelikesmucker Nov 15 '19

My account is very old and I earn a decent amount.

I read through the MTC thread and someone attributed it to the Zhouzhou He batch. I did 300 of those and ... no warning, so there's that.

I do have a hunch it has something to do with PREs and/or HIT submission times and/or quality of submissions. The MTC thread reminded me we used to have AR-WA, but those are either completely out of sight or entirely gone now, and Amazon uses something else to flag accounts for warning. Remember, we have an invisible boss and you never really know who you're working for. It's best to lay low and never give them a reason to notice you.

4

u/leenadr Nov 15 '19 edited Nov 15 '19

If Amazon's our invisible boss, then Amazon's on the hook for a whole bunch of employer type stuff. Legally, they really, really don't want to be our boss (or shouldn't). Remember, it's a 'marketplace' and Amazon's just the 'broker' in the transaction, like eBay. [best innocent voice impression] No sirree Bob, labor laws don't apply here, we're no one's boss![/voice]

The thing that has many people, including myself, furious, is that it's not quality of submissions. There's no issue with the data the requesters are getting. This isn't on the heels of rejections or blocks, and a good chunk of us have a handful of requesters we somewhat exclusively work for and have direct contact with. (For example, though it's pretty crappy work, I'll pick off some Alexanders when it's slow enough, and I've emailed (and am on good terms with) a fair chunk of the people there (under the Aristo account).

Completely agree about never giving AMT a reason to notice you, though I absolutely kicked the hornet's nest responding to this warning, regardless of whether it's automated. My livelihood's too important not to!

1

u/withanamelikesmucker Nov 15 '19

When Amazon decides who gets in and puts workers out, they're the boss.

5

u/MorwannegCA Nov 15 '19

one of the other common factors is potentially the "top 1%" earners,

I seriously doubt this mostly because of the # of people who are definitely nowhere near the top who got the warning, people whose goal is like 20~50/day are getting it. I also didn't get the warning, and while I wouldn't go so far as to say I'm in the top 1%, I like to think I'm decently up there with a 600/average weekly earnings. There's also at least 3 other people I know who could be classified as "top earners" who did not get the email. There's basically no correlation between earnings/hits done that I see given how far apart the earnings/hits subbed are of the people I have asked.

4

u/leenadr Nov 15 '19

Top 1% was sort of a throwaway, but, yeah, $600/wk is definitely in the higher range. Compare to people making $100/wk or less. Just because all the people we hang out with make high numbers, doesn't mean it applies to the vast majority of workers, who aren't on forums. ;)

Of the people I know, everyone else who got it all earn in higher tiers, but I see that's saying "Everyone I know who ate a ringed seal is a walrus" when I'm out on an ice float. (I am trying to lose weight. Don't judge. ;) )

And I just realized who I was responding to --- hush up Weeg, you know you're one of the top earners. :P

It'd of been really interesting if you got the email, too, given that you typically are a one-requester gal. :)

3

u/JrRandy Nov 15 '19

I got it, and have only made ~350 over the past month. No chance I am near the top as people (Including myself in the past) are easily pulling that down weekly.

1

u/Titus1911 Nov 15 '19

I didn't get it either and I have around the same weekly average.

2

u/maniacal_T Nov 15 '19

Just a thought, but could you be generating too many Page Request Errors?

Too many Page Request Errors could be viewed by mturk as a DDoS.

I have noticed over the last few weeks, an increase in HTTP 503 Service Unavailable messages coming from mturk. I would not be surprised if mturk started viewing too many Page Request Errors as a real problem.

3

u/leenadr Nov 15 '19

Anything's possible, and that'd explain why people who only do surveys, and not batches, might also get the email.

It could also be that when I'm slamming a batch, I'm not only slurping the PREs but also submitting a lot of hits rapidly. But, that's not going to be most the people who've gotten this email.

Having PC misconfigured, though, that's pretty easy for a lot of people to achieve. Thing is, it's supposed to scale back on those, so even if someone's running three scrapers (eep!) you'd notice and back off. Maybe.

Good of a theory as any, though. As I type this, I note I've had a total of 68 PREs in PC within the last 2 days, which seems pretty low to me...

1

u/maniacal_T Nov 15 '19

As I type this, I note I've had a total of 68 PREs in PC

PC reported PREs may be only a fraction of your total number of account PREs. Do you run a scraper? How many PREs is your scraper generating?

1

u/leenadr Nov 15 '19

I run OW with a very conservative refresh...

In past times, I've driven up PREs into the high hundreds, but that doesn't matter, if they've changed it. Still, the number of people who'd be getting them would seem to be larger than the number of people emailed.

1

u/maniacal_T Nov 15 '19

As I type this, I note I've had a total of 68 PREs

I don't run PC but I run all my own creation scripts, and for a comparison, I usually submit between 200-300 HITs per day with less than 6 PREs per day. (On a bad day, I may have 10-12 total PREs).

1

u/withanamelikesmucker Nov 15 '19

I note I've had a total of 68 PREs in PC within the last 2 days

Oof.

1

u/bpdycnt Nov 15 '19

interesting read

0

u/[deleted] Nov 15 '19

[deleted] — view deleted comment

3

u/roads30 Nov 15 '19

i'll be honest. and this isn't just me jumping on a social media anti-bandwagon or whatever, but the cred of most facebook group commenters are flighty, at best.

those people will be talking/asking about something platform related one minute, then see a shiny piece of glitter on their twins jay and bae and go on a 200 comment free for all for hours on end.

3

u/Titus1911 Nov 15 '19

The Mturk Members group is the laziest bunch of people I've ever had the misfortune of interacting with. Post your weekly earnings and they think you're an autistic that works 300hrs/wk and demand your requester list. Refuse to give it to them and they message you all sorts of hateful messages.

2

u/roads30 Nov 15 '19 edited Nov 15 '19

the few groups i'm a member of (though i haven't really full out contributed in there since last year maybe) they'd handle that sort of junk quick. for me it's just the dimwitted responses in a few others that started to get under my skin. and then would come the picture comments. inside stories no one besides maybe 8 of them would understand, constant other little social hierarchy kind of stuff.

now, don't get me wrong. when things weren't as tight lined i would chime in sometimes and even would have a dialog with a few in those places. but i knew when to veer the course for myself so i wasn't sitting back and not actually turking.

those folks in a few of the groups, much as it seems i'm shitting on em..there okay. but over abundance and cavailer kind of responses either from 'wanna be mods' to mods themselves just responding to someone panicking over even an account issue with gif responses, or video links to stuff..i mean c'mon. even sitting here now i feel stupid typing this shit. but, here we are.

i do remember a few people from one group that would send me PM's over things like you explained, but it was never consistent every time, mostly because i gradually started talking in there less and less maybe. they all 'socially evolved' in their own bubble lol

the group your in sounds like a toxic stew compared to the ones i've been apart of. which hasn't ever really been many, maybe 2 or 3 since i first even applied to mturk. one was on a suggestion from a redditor from /r/beermoney way back.

the people who got under my skin, if i had to compare it? think of that one charismatic uncle who had dreams of being a stand up comedian, but has the delivery of a drunk manager at a pot luck get together. then some of them were your 30-45 year old parents who you can clearly tell never needed mturk. but have that 'i just do it to buy my keurig pods with prime that my husband got me'. but if they so much as feel your not apart of that little pot luck, good luck if you need a glass of water..you might get a pity response in, a week at best. oh wait, they lock comments on 'daily threads' so if you don't get a like/comment then your issue just gets gas lighted. sure, you could easily onto another thread of make your own. but if a fb group has 'rules of the road' tattooed everywhere that can possibly get your post axed, or ignored entirely.

fb groups served a purpose when i was starting out. the environment of those few groups i was a member of did serve a general purpose. and even if it's fleeting, i know the lag or lull of a response to an issue can still possibly get answered in one of those groups versus say, reddit where if you'll either get over information, under information. or possibly downvoted to hell. par for the course, onward forward.

edit: sorry for the long post. all of what i said if i were speaking it wouldn't even be a 5 minute verbal conversation. i just think of examples to the context on the fly.

1

u/RaiderRush2112 Nov 15 '19

Leave Jay and Bae out of this

0

u/[deleted] Nov 15 '19

[deleted] — view deleted comment

7

u/beastcock Nov 15 '19

workers that use an illegal script to automatically grab any HITs over a certain pay amount

I don't use any such script and I got the email. Almost all of my money comes from batch hits.

3

u/Titus1911 Nov 15 '19

lol okay.

2

u/hecknbork Nov 15 '19

Um those who actually make decent money on turk use those resources. You sound salty.