The Mechanical Turk Blog

Mechanical Turk Workers Help MIT Scientists Identify Endangered Species

If you watch animal documentaries or TV shows on Animal Planet, you may be familiar with how scientists use physical tags to track animals in the wild. However, sometimes tagging can be difficult or intrusive on an animal’s environment. A recent article from LaboratoryEquipment.com highlights how scientists at MIT are using photographs to track animals, along with algorithmic pattern recognition and Mechanical Turk Workers to identify animals in the images collected.  

While capturing photographs is less intrusive, reviewing the shots collected is an arduous task. To keep scientists in the field (as opposed to manually reviewing images in front of a computer), Ravela’s team developed a software system that uses algorithmic pattern recognition to identify animals based on their unique features, such as stripes or spots. 

Ravela’s team was able to successfully identify potential image matches algorithmically, so the team turned to Mechanical Turk to create a scalable process for manual verification.  To identify the most qualified Workers for the task, each Worker was presented with four images, where the correct answer was known for three.  If the Worker correctly matched the known images, then the system would automatically trust the accuracy fourth result. 

Read about MIT’s experiment at on LaboratoryEquipment.com.  To learn more about how you can use Known Answers to monitor performance, see our previous post on the topic or read more in our Mechanical Turk’s API documentation.   

The “Smarter” Way to Encourage Recycling

If your friends could observe every item you decide to toss in the trash – as opposed to the recycling or compost bin – would it influence your behavior? A group of designers based in the UK and Germany think it could. To prove it, they created BinCam – a “smart” trash bin that tracks the items you toss, distinguishes between responsible and irresponsible behavior, and broadcasts your performance to your friends.

How does it work? While it looks like an average trash bin, the BinCam is equipped with a smartphone that captures an image each time the bin is closed. The image is then posted to Mechanical Turk where Workers review the image and determine whether or not the decision to trash the item pictured was responsible or irresponsible based on recommended recycling practices. For responsible decisions, BinCam users earn points. Alternatively, for making poor decisions, users are penalized by having the photographic evidence posted to Facebook for all to see.

To learn more about the BinCam and read broader commentary on the potential merits (and dangers) of smart technologies – check out Evgeny Morozov’s recent essay in The Wall Street Journal.

 

(image credit: Enadget)

The Geography of Happiness – As Measured by 10 Million Tweets

In a new study published by the Vermont Complex Systems Center, researchers investigated correlations between the content of tweets from across the U.S. and their geographic origin.

The researchers used Mechanical Turk to rate the sentiment of 10 million geo-coded tweets collected over the course of one year. For instance, words like rainbow, love, beauty, hope, wonderful, and wine were considered by Workers to be happy; whereas words like damn, boo, ugly, smoke, hate, and lied are examples of words considered to be sad.

So, what city was determined to be the happiest? Apparently, Napa, CA is the happiest city in the country. By contrast, Beaumont, TX is the saddest city. Louisiana was determined to be the saddest state. Maybe not surprisingly, Hawaii was rated as the happiest state.

To learn more, check out The Atlantic’s recent coverage of the study and their commentary on results. For more detail - including more analysis on how the tweets correlated with a wide range of emotional, demographic and health characteristics – read the entire study online at arXive.org.

 

FIG. 1: Choropleth showing average word happiness for geotagged tweets in all US states collected during the calendar year 2011. The happiest 5 states, in order, are: Hawaii, Maine, Nevada, Utah and Vermont. The saddest 5 states, in order, are: Louisiana, Mississippi, Maryland, Delaware and Georgia. 

 

Mechanical Turk, Big Data and the Impact of the Internet on Social Science Research

“The internet is transforming how researchers perform experiments across social sciences,” according to Walter Frick, a guest contributor to MIT Technology Review. At Mechanical Turk, we see evidence of this statement every day based on the growth of social science research conducted using our marketplace.

Frick recently attended MIT Technology Review’s EmTech sessions on big data. In his article covering the event, Frick describes how the internet is expanding our notion of the “lab” by highlighting examples of how some researchers are leveraging Mechanical Turk, while others are using big data generated by sites like Facebook to conduct social science experiments.

While these two approaches to conducting online research are seemingly contradictory, Frick contends that they are actually complimentary. He makes the point that in order to ask the right questions of big data, you need the theories generated by social science experiments. Mechanical Turk makes this sort of experimentation more accessible due to the cost effectiveness of collecting data through the marketplace.

To read more about the research presented at EmTech and Frick’s takeaways, read his full article on MIT Technology Review online.

Mechanical Turk for research studies

A recent article in Science profiled how social scientists are using Mechanical Turk to conduct research studies.   Mechanical Turk lets scientists test and conduct studies quickly and cheaply and choose participants based on specific criteria from over 500,000 Workers in 190 different countries.

The article details how scientists are using Mechanical Turk to conduct bigger, faster and cheaper research studies.   For example, social scientists used 10,000 Turkers (as Mechanical Turk Workers are sometimes called) to create a tool for tracking the emotional content of Twitter messages (Science, 30 September, p. 1814).  In another study, Gabriel Lenz, a political scientist at the University of California, Berkeley, asked Mechanical Turk Workers to look at photographs of Brazilian political candidates and fill in a data sheet.  According to the article, Lenz and his team had all the data he needed in just a few weeks and at a total cost of $160. The results of their research appeared in the social science journal World Politics.

"The use of Mechanical Turk subjects will eventually become mainstream" Adam Berinsky,  a political scientist at the Massachusetts Institute of Technology, says in the article. Berinsky goes on to predict that many more social science papers using Mechanical Turk will appear.  “Everyone I know is using it,” he says.  "The obvious advantage is the speed and cost."

You can find the article here.

Get Better Results with Less Effort with Mechanical Turk Masters!

We're excited to introduce Mechanical Turk Masters. Masters are an elite group of Workers who have demonstrated superior performance while completing thousands of HITs across the Marketplace.

By publishing your HITs to Masters, the quality of your answers will increase (you’ll get right answers more often) and managing results and Workers will be easier. You may even find that you can publish fewer assignments per HIT and still get increased accuracy.  Mechanical Turk Masters provide Requesters with an easier way to distribute HITs to Workers who have proven themselves on the Mechanical Turk Marketplace. 

Starting today, Requesters will be able to access two types of Masters -- Photo Moderation Masters and Categorization Masters.   Workers who achieve these distinctions are granted the corresponding Masters Qualification for either Photo Moderation or Categorization.

Although you can send any type of HIT to Mechanical Turk Masters, Workers who have been granted a specific Masters Qualification have demonstrated superior performance at these types of HITs. For instance, Photo Moderation Masters are the best choice if you want to have a photo reviewed to determine if it meets your site guidelines, or need specific types of content identified such as the number of people in a photo. Categorization Masters are the best choice if you need to have companies categorized by industry or identify the department for products in a retail catalog or characterize the sentiment of a tweet or blog post.

Getting Started with Mechanical Turk Masters
Accessing Mechanical Turk Masters is easy. Requesters simply require Workers have one of the Master Qualifications when creating HITs via the Requester web site, API or Command Line Tools.  Requesters can further refine their workforce by requiring Workers have other system or custom Qualifications in addition to the Masters Qualification.

Congratulations Mechanical Turk Masters!
Achieving Mechanical Turk Master status is an honor for Workers – and we want to congratulate the Workers who have achieved this distinction! Kudos!

Net:Work Conference Recap

Last week Sharon Chiarella, VP Mechanical Turk and I attended the GigaOm Net:Work Future of Work conference in San Francisco. These events are so awesome – it is great to spend time with the Crowdsourcing community, our customers and partners. We hear such useful and insightful feedback. 

Sharon participated on The Human Cloud: Elastic Workforce in the Enterprise panel along with Alex Edelstein, CEO of CloudCrowd, Maynard Web, CEO of LiveOps and Doron Reuveni, CEO of uTest and was moderated by Niel Robertson, CEO of Trada. 

The panel was asked to provide insight into the value of labor-as-a-service within the Enterprise, the impact of Cloud-based labor on outsourcing and how the category has changed over the last few years.

Some key highlights: 

• Platforms like Mechanical Turk are speeding up the pace of innovation by allowing businesses to get work done more quickly and scale up and down with an on-demand labor pool as their needs change. 
• Anonymity of Workers was discussed. Alex Edelstein had a great parallel about the importance of a Workers identity – do you need to know who installed the steering wheel in your car to know if the work that was done was high quality? Sharon agreed that it is not an individual’s identity that’s important but the quality of their work. 

Watch the video below to get the full details from the panel. Thanks to GigaOm for hosting such a great event – we were glad to be a part of it! 

-- Natala 

 

Hello world!

Welcome to the official Mechanical Turk blog!

We’re launching this blog to share information and updates with Requesters, Workers and fans of the Mechanical Turk marketplace. We plan to provide you with useful information such as product announcements, how-to guides, best practices, API tips and tricks as well as case studies, interviews and examples of how businesses are using Mechanical Turk for everything from transcription to content moderation to data cleansing and search algorithm testing.

You can also stay updated by following us on Twitter @amazonmturk or friending us on Facebook.

The Mechanical Turk Team

	Follow us on Twitter
	Become a fan on Facebook
	Subscribe to our RSS feed