Over the last month, The Mechanical Turk team attended three awesome conferences where we got to meet with a variety of people and answer questions about Mechanical Turk. It is always great to hear about the things people are doing on the platform and answer your questions. Here are some of the presentations and a short recap of the conferences.
CHI Conference – Crowdresearch Workshop
In partnership with the CHI, Computer Human Interaction conference, an amazing group of Academic leaders put together a one-day Workshop focused on Crowdresearch. To participate in the conference we submitted a paper on an area that we felt warranted additional research. Our paper focuses on the need for studies in the areas of HIT design, workflow and reviewing methodologies, and compensation strategies to establish a standard library of repeatable patterns and protocols that achieve high quality, consistent results for a variety of human computation tasks.
In the paper we dive into the key areas where Requesters can explore the system and strategic levers that impact result output such as:
HIT Settings
- Time (available, to submit, to approve)
- Price
- # of Assignments
Worker Qualifications
- Location
- Skills (via experience or testing)
- History (time on platform, approval rate)
HIT Design
- Ergonomics (impacts on velocity and efficiency)
- Instructions (guidelines for how to do the task)
- Defensive Design (design to get the results you want)
Adjudication
- Bonus Rewards
- Forced agreement
- Plurality
- Expert Review
- Known Answers
You can read the entire paper here – or browse all of the papers that were presented here.
Crowdsortium
At the Crowdsortium conference leaders from a variety of “crowd” related companies came together to discuss issues and opportunities faced by this type of work. It was a great experience to spend time with so many people thinking about similar problems of Workforce growth, management and skills. If you are a professional working in Crowdsourcing, check out the Crowdsortium.
SMX Advanced – SEO Power Tools Session
In this session, we presented information on how Mechanical Turk can be a power tool for SEOs. If you’ve ever had to manage classifying or expanding a large data set of keywords, this session was for you. The presentation below outlines some of the key use cases for SEO and SEM professionals – check it out!
We hope to see you at an upcoming conference!
Comments