|
Turk work
Through the lens of the podcast transcription service CastingWords, I've observed
Amazon's MTurk from
a few different perspectives. In a podcast
conversation with CastingWords' Nathan McFarland and Mycroft's Ben
Hill, we explored network-enabled ways of harnessing collective
intelligence. That podcast was then transcribed by CastingWords, and I reviewed
the results. More recently I became a satisfied
CastingWords customer.
Last night, to complete the picture, I joined the global workforce of
Turkers -- that is, MTurk-enabled workers. It's a weird subculture
that Katharine Mieszkowski explores in her excellent Salon article
I
make $1.45 a week and I love it.
One of my talks last week focused on the idea that blogging can open
windows into the world of work. That's certainly true of Turk
Work. Blogs such as Turk
Lurker, Mechanical
Turk Monitor, and CastingWords Turker
chronicle the experiences of Turkers as they churn
through HITs (Human Intelligence Tasks), rack up pennies, exchange
tips for finding and optimizing work, and reflect on the often bizarre
nature of tasks that, at least for now, do not yield directly to automation.
Here are some of the HITs available to me at the moment:
The description of the third HIT reads:
To make one's mother to stand on one foot (with the other one
lifted) and to picture this outstanding scene. Name of the mother, her
age and the location are needed. (TIME!!!) The fee - $2 per chosen
picture.
People in online sweatshops who are paid to
to play games in which they create virtual artifacts, earn
virtual currencies, and build virtual reputations have fallen down one
kind of rabbit hole. Many Turkers have fallen down another, as shown
most strikingly by the Aaron Koblin Sheep Market project
described in Mieszkowski's Salon article.
CastingWords, of course, is a very real and very effective
business. So I did a bit of transcription in order to see what it's
like from the Turker perspective.
Podcasts are chopped up into six-minute segments. Last night, the
segment I transcribed was worth $1.02, which will be credited to my
Amazon.com account (and can then be transferred to a bank account) if my work
passes several quality review checks. These checks are themselves implemented
as HITs.
This morning, segments from that same batch are worth $1.28. (I should
have waited!) At that rate, a Turker who could transcribe accurately
and in realtime would earn $12.80/hour for himself and $1.28 for
Amazon. I can't do accurate realtime transcription, of course, so I'd
be lucky to approach minumum wage.
How'd I actually do? To be honest, I lost track. The segment I chose
at random turned out to be an interview with Mauria
Aspell, a childhood friend of Bill Clinton who dated him in high
school and college, and who was evidently being interviewed for a
history of Clinton's relationships with women. Thirty seconds into the
recording I guessed what it was about, but when the name
finally dropped it was still a shock. Bill Clinton's romantic past, in 6-minute
segments, parceled out to the Turk Nation for transcription at $1.28 a
pop. Whoa. Where's
my blue pill?
Celebrity dirt notwithstanding, my encounter with Turk Work wasn't
very satisfying. There was no context, no orderly progression, no sense
of collaboration, no awareness of (or pride in) a finished product.
The Salon article suggests that these are essential qualities of the
experience. And as I look more closely at how MTurk is structured, I
suspect that's right.
If you're running an MTurk-enabled business, you have to focus on
throughput and efficiency. That drives you toward assembly-line
tactics. But as Nathan McFarland noted in our conversation, you'd also
like to reward excellence:
We'd be willing to pay certain workers more and other workers less,
but that's not an option right now.
So for now, that's a hard constraint. Here's another. From my
perspective as a CastingWords customer, there are clearly some
transcribers who have a better feel for my material than others.
I don't know who they are,
though, or who's transcribed which parts of various podcasts. If the
goal is maximum throughput, that may be necessary. But that's not my
goal. Transcribing 620 minutes
of audio for $260 in six days was darned impressive, but two
months later I'm still only halfway through the final polishing, which I'm
tackling in fits and starts. If this winds up being a long-term
relationship, I'd rather identify the transcribers who do well on my
stuff and give them whole tasks. And some transcribers
would likely prefer that too. I sure would. Apart from the question of
whether it's in CastingWords'
interest to allow such relationships to form, the current MTurk
architecture precludes the possibility.
There are, of course, other architectures. Last week in Ann Arbor, for
example, I met Bill Tozier, who among other
things works for Distributed
Proofreaders, an adjunct to Project Gutenberg. "We're the
prior art for MTurk," Bill told me.
MTurk is one model for the online coordination of work. There will be many
others, some commercial and some open,
each constituted in its own way.
One final thought on MTurk. When the tsunami hit, and then Katrina, I
used Amazon in both cases to contribute to disaster relief. It wasn't
very satisfying, though. In retrospect I'd rather have contributed
work to the Katrina
People Finder. Next time around, it'd be cool if Amazon used MTurk
to coordinate that kind of work, and really cool if it found ways to
create social capital in the process.
|
NEWS 
