In order to collect insights from Workers, we posted a HIT to Mechanical Turk asking Workers to share their perspective on actions Requesters can take to make their HITs more attractive. Based on previous surveys we’ve conducted, we know that a primary motivation for Workers is monetary reward. However, in this exercise, we wanted to explore other approaches that Requesters can use to build their relationship with Workers outside of compensation.
So, what did Workers have to say? In reviewing their feedback, three key themes emerged:
Communicating with Workers by addressing questions and providing feedback can help identify issues in HIT design and improve the quality of results.
“Communication! A great requester communicates regularly with their workforce. That means answering emails promptly or connecting with us in the forum to address questions and concerns, announce changes and upcoming work, explain work and rejections policies in more detail, etc. Communication makes everything easier, for everybody involved. Nothing worse than being rejected or even blocked and never [knowing] why.”
“The great requestor gives feedback for good work and bad. If there is a way that a worker can improve their work, constructive feedback is the only way to know how to do so. Good feedback inspires more good work.”
“Requesters who take the time to communicate are at the top of my list. Communication allows the workers to have a better idea of what the requester is looking for and in turn provides the requester with better work. It doesn't take much just an email or two and has an enormous return on investment.”
"An open line of communication between Worker and Requester can improve any issue on either side. Being open and responsive to feedback is a great sign that a requester is interested in improving the quality of work and the quality of worker/requester experience on MTurk in general.”
“Having an active presence on [a Mechanical Turk Forum, like TurkerNation] affords a Requester the opportunity to be a great Requester. The ability to communicate with a group of Workers, as well as recruit new Workers, is a time saving, cost-effective way to advertise tasks, clarify or modify instructions and/or qualify Workers. A Requester that is willing to do these things is not only a great Requester, but a great Requester [who] will obtain quality results on the tasks they publish because the Workers will be waiting for their HITs to launch.”
Approving (or Rejecting) assignments in a timely manner helps Requesters build trust with Workers.
“Pay promptly. If you have a lot of HITs and you are a new Requester to me, I will be reluctant to do a large batch of hits before I know that I will get paid. If I start doing a batch of HITs for you, and within a few hours I see "payment pending" (meaning that you’ve approved my work) for the first 100 or more HITs, I will have no reservations in spending the entire day and the next day working on your HITs.”
“I prefer hits that [are approved] quickly. It helps me to judge my work. If there is a problem I can fix it if need be. Plus getting the money quickly helps. Some HITs take days to get approved. By then I have moved on to other hits so if there is a problem it is hard to fix."
Mistakes happen, but quickly reversing mistakenly rejected assignments can encourage Worker loyalty.
“Great Requesters know when to admit that they have screwed up. I've had many times where I felt that my work was unjustly rejected after I had spent a good amount of time on it. When I contact great Requesters about this type of issue they find a way to make it right, either by offering me a bonus for the rejected work or additional work that will earn back my trust and respect for them.”
“Learning how [Mechanical Turk] works, specifically reversing rejections. Rejections stay with a Worker forever even when the requester is at fault. When a Requester learns how the UI works and realizes they can reverse rejections, I'm more likely to stick with them on future tasks.”
“Quick and honest dispute resolution. On several occasions I have had work incorrectly rejected. When I have reached out to the Requester, they have responded quickly by double checking the error in question and either admitting the mistake on their part or acknowledging that the reject was in error. They either reverse the rejection or they [issue a] bonus...”
To learn more about best practices for achieving results on Mechanical Turk and building your reputation with Workers, be sure to review the Requester Best Practices Guide.