We wanted to take a moment to share a few exciting and recent enhancements to the Categorization Application that make it easier for Requesters to analyze results and reuse previously created templates.
New Results Review Carousel
First, we’ve launched a new feature enabling Requesters to review Workers’ results in the context of the HIT as it was presented each Worker. By reviewing results in this way, Requesters can more thoroughly understand the reasoning behind the Workers’ results and discover potential opportunities for improvement.
For example, if you’re using the Categorization Application to categorize images, you now have the ability to review the image and the instructions alongside the Workers’ answers. Reviewing results in this way, as opposed to spreadsheet format, can be helpful in understanding potential causes of Worker disagreement, or alternatively, understanding why Workers agree on a result.
As a Requester, you have the ability to review all results in the carousel or filter results based on whether or not Workers agreed or disagreed (see example of filter options in the image above). In some cases, reviewing disagreements and agreements can expose opportunities to clarify instructions and subsequently improve the quality of results on future batches.
Access the results carousel from the “Manage” tab. Once there, select the project you wish to review and then click “Results” under “Categorization Project” in the left-side navigation. Click the arrow on the right of the result image to scroll through individual HIT results.
Reuse Projects Created with Categorization App
Additionally, you can now launch new batches using previously created categorization projects. Reusing projects was not previously possible for projects created with the Categorization Application.
To launch a new batch using an existing project, select the “Create” tab to get started.
Next, click the “New Batch” button next to the appropriate project (pictured above) under “Start a New Batch with an Existing Project”. If you would like to make edits to your project, click “Edit” to make changes to the HIT properties or design layout.
We hope that you find both of these enhancements to be helpful and that they make the process of reviewing results and launching new batches on existing projects easier and more efficient.
--The Mechanical Turk Team
Comments