This week, we'll be discussing the merits of Amazon's Mechanical Turk as a means of data collection, as well as continuing your class projects.
(1) Be a Turker.
Please sign up as a Mechanical Turk "Turker". Perform about 30 minutes of "HITs" - human intelligence tasks. You can do many short tasks or a few longer tasks - it's up to you.
Answer the following questions:
1) Record how many experiments you did. How much money did you earn?
2) What kind of tasks did you find yourself drawn to?
3) Did you find any tasks to be confusing? Boring?
4) If you were (or are) designing a task for AMT, how would you design those tasks to obtain the best results?
(2) AMT Reading.
Now that you have a feel for what Mechanical Turk is about, read this paper by Snow et al: Cheap and Fast - But is it good?. The paper discusses the use of "crowdsourcing"via Mechanical Turk in NLP.
Please write a 1 paragraph summary of your thoughts on the paper. What do you think about their methods? Do you think the NLP domain is "well suited" for AMT? (And, after, our reading last week, what do you think about their use of graphs?)
3) Statistics reading.
We think you'll enjoy the first chapter of Robert Abelson's Statistics as Principled Argument. (This was emailed to you - please contact the TA if you still need a copy.) Please write a paragraph about which of the 5 "MAGIC" criteria you think will be the most important to fulfill when forming a persuasive argument about your project.
(3) Continue working on your projects.
On your blog, make a brief but thorough checklist of everything that needs to happen between now and your final presentation. Be specific. This might include things like:
What infrastructure details do
you need to take care of? (code to get working, props to arrive, ...)
How many subjects will you need to recruit?
How much time will you need for your experiments, and when will you schedule them?
How many subjects will you need to recruit?
How much time will you need for your experiments, and when will you schedule them?
(... etc. )
Rank each task in the order in which you intend to complete it, and for each item, include a deadline. The goal here is to help you plan the next few weeks and to foresee potential bottlenecks.
Please note that on 5/13, we'll be having another project "Checkpoint" in class where you'll give updates on your progress. If applicable, please bring prototypes of your experiment to show the class. Is there anything you've tried so far that hasn't worked out? That's important too!
Preliminary data and graphs will be presented in class on 5/20.
By 5/27, we'd like everyone to have made thoughtful, constructive comments on at least 6 of the blogs of the rest of the class. Bonus points for posting on all 8 of your classmates' blogs!
(0) Answers to the questions in
part (1).
(1) Review paragraph for the Snow reading.
(2) MAGICal paragraph linking the Abelson reading to your project.
(3) Blog post with project updates and checklist.
(4) Let me know whose blogs you've commented on so far.
(1) Review paragraph for the Snow reading.
(2) MAGICal paragraph linking the Abelson reading to your project.
(3) Blog post with project updates and checklist.
(4) Let me know whose blogs you've commented on so far.