« Net:Work Conference Recap | Main | Search by Requester Name »

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a013488051a84970c0147e0ce740d970b

Listed below are links to weblogs that reference Some thoughts on invalid HITs :

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

shannon

I am finding that that the reporting HITS that violate the PA of your Site they end up being taken off but put right back up as soon as the workers are reporting this information to you guys. These Requesters are continually renewing and coming back again and again into the system. I am wondering how this is being closely monitored.

anon

Amazon, you do exactly zero pro-active work to eliminate bad requesters. Evidence that you do nothing to remove scammers unless requested can easily be seen by sorting all HITS by reward amount high to low. All of the highest paid HITs are scams and are quite easy to find and you have not removed them.

(And that doesn't even include the time you let someone solicit tales of child abuse I though were solicitation for child pornography.)

Mechanical Turk

@shannon - it is definitely a tough game managing aggressive invalid HIT requesters. We unfortunately can't speak specifically about our internal methodologies, but we do monitor this activity at a very granular level.
@anon We've made a few modifications to improve that sort specifically, have you noticed an improvement?

spamgirl

Easy answer to managing aggressive invalid HIT requesters - change policy to require a $100 payment upfront which is not refundable. Tack on increasing ID checks for requesters (driver's license, phone check, credit card number on file, etc.) and you'd be good to go.

Broken turk

Did you change something in the past few days, since Panos Ipeirotis published his review of spam on mTurk? Can you explain how that change was implemented, or rather, why it took so long to do something that changed so quickly?

J. Yendriga

I wouldn't claim success for getting rid of the spam HITS quite yet.
The is still being posted:

Join this free website and report the feedback

This is a very short sign up job for you to take. It is a very fast job that can be done in under 1 minute! You may only take this HIT once.

* Go to our website here
* Complete the sign up process of the first 4 pages with your real information.
* Once you receive the thank you page you are finished. If this has gone wrong you need to try again.
As part of the survey it is possible you will be contacted. Please answer the person as normal as possible as he/she is also doing her job to follow up your online request.
Do not reveal you take this as a job but ask for more information, or tell that you have changed your mind.

And many more from the requester mrutknews on this December 26 morning. Also, like before reporting them does not get them removed

Jimfribly

I have noticed a TINY difference.
It really is difficult when you continually see 7 or 8 hits that you have already reported still on screen hours/even days later.
You know a newbie is going to fall for it.
So many are CLEARLY a violation of TOS, that even a turker who has only been working on the site a week can see.
Oh well, thanks for trying Amazon, at least we know someone is trying to bring MTurk back to what it used to be before the spammers found it.

Mechanical Turk

Hey everyone -- just wanted to let you know that we are listening to your feedback -- and we really appreciate it.

The comments to this entry are closed.