Amazon's Mechanical Turk 375
rscoggin writes "Amazon.com has a new program that wants you to 'Complete simple tasks that people do better than computers. And, get paid for it.' (example: 'Is there a pizza parlour in this photograph?'). For each task you complete you get a small payment, usually ranging from a few cents to a little under a dollar. It's named the Amazon Mechanical Turk after a famous hoax from the 19th century. Kill time and get paid in tiny increments to boot!" Similar to Google Answers, there seems to be a reliability ratings system and some incentives.
Great... (Score:5, Funny)
GOOD JOB AMAZON
Re:Great... (Score:5, Funny)
Micropayment mercenary (Score:3, Funny)
Re:Micropayment mercenary (Score:3, Insightful)
"Micropayments" are used a lot of send small payments to 'charitable' organizations--not just bona fide charities, but things like Open Source projects.
Suppose you could sign up and do this, but have the proceeds sent to charity? Getting 3 cents isn't at all useful for me. If many people sent their 3 cents to a developing country, it would matter.
I can't navigate the site that well right now (everything's returning errors), so I'm not sure if this is in the works or
CAPTCHAs (Score:3, Interesting)
Re:CAPTCHAs (Score:5, Informative)
Subject: Thesis Oral - Luis von Ahn
November 2, 2005
Luis von Ahn
12:00 PM, 3305 Newell-Simon Hall
Thesis Oral
Title: Human Computation
Abstract:
Tasks like image recognition are trivial for humans, but continue to
challenge even the most sophisticated computer programs. This thesis
introduces a paradigm for utilizing human processing power to solve
problems that computers cannot yet solve. Traditional approaches to
solving such problems focus on improving software. I advocate a novel
approach: constructively channel human brainpower using computer games.
For example, the ESP Game, introduced in this thesis, is an enjoyable
online game -- many people play over 40 hours a week -- and when people
play, they help label images on the Web with descriptive keywords. These
keywords can be used to significantly improve the accuracy of image
search. People play the game not because they want to help, but because
they enjoy it.
I introduce three other examples of games with a purpose: Peekaboom,
which helps determine the location of objects in images, Phetch, which
collects paragraph descriptions of arbitrary images to help
accessibility of the Web, and Verbosity, which collects common-sense
knowledge. I also show that, in principle, every problem that could be
solved by a computer, today or in the future, could be solved using
enjoyable computer games.
In addition, I introduce CAPTCHAs, automated tests that humans can pass
but computer programs cannot. CAPTCHAs take advantage of human
processing power in order to differentiate humans from computers, an
ability that has important applications in practice.
The results of this thesis are currently in use by hundreds of Web sites
and companies around the world, and some of the games presented here
have been played by over 100,000 people. Practical applications of this
work include improvements in problems such as: image search,
adult-content filtering, spam, common-sense reasoning, computer vision,
accessibility, and security in general.
Thesis Committee:
Manuel Blum, Chair
Takeo Kanade
Michael Reiter
Josh Benaloh, Microsoft Research
Jitendra Malik, University of California, Berkeley
Re:CAPTCHAs (Score:2)
Re:CAPTCHAs (Score:3, Funny)
i can see it already... (Score:5, Funny)
- Is there a donkey in the picture ?
- Can you see the can of whipped cream ?
- is there chocolate paint involved..
Advanced indexing of Pr0n, humanity is moving forward, no doubt.
Re:i can see it already... (Score:2)
This could be brilliant. (Score:5, Insightful)
Re:This could be brilliant. (Score:2, Interesting)
Re:This could be brilliant. (Score:4, Funny)
Re:This could be brilliant. (Score:3, Interesting)
Question: What kind of pop/soda do you see in this picture:
First picture shows a party with several attractive men and women and [good soda].
Second picture shows a senior care hospice and one can of [bad soda] being shared between 5-6 individuals.
Third picture shows a couple in a fast car along a mountian pass, each holding [good soda].
Fourth picture shows a prison cafeteria with a badly maintained [bad soda] machine.
I can't wait for the next election. This kind of "advertisin
Re:This could be brilliant. (Score:3, Informative)
So this is going mainstream now... (Score:5, Informative)
According to this earlier Slashdot report [slashdot.org], the spam industry has been doing this for awhile with free porn.
I'm curious to know if Amazon is going to use the cumulative results to try to "train" computers, or if it really is just for the money. The requirements include being over 18, so you can't pimp your kids to click through this stuff for cash (though I'm sure it will happen).
A new kid-powered architecture! (Score:3, Funny)
Or maybe this would require a new "Grid Kid" architecture with an advanced resource broker to farm out the questions based on difficulty and school grade level.
Doesn't pay enough (Score:5, Insightful)
Maybe it's just me, but it seems that it's not really worth it. Consider the following task, for example:
Guess how much you get paid for that. 2 dollars? 3? That wouldn't be unreasonable, I think, considering that you're supposed to write an entire product description from scratch for which additional "research" is required. The actual amount paid is only 65 cents, though.
Maybe it's just me, but if I check to see how much I need to work in my regular job to make 65 cents, then it does not make any sense to invest more than a few minutes into a task like this, and it seems that it would take more than that to actually complete it. The fact that there's a review required afterwards doesn't exactly make things better, either - if what you did gets rejected, then you've essentially worked for nothing (I wonder if there's anything that keeps amazon from still using your description in this case, too...).
In other words, the whole thing seems like a good idea in theory, but it won't really take off until they're willing to actually pay you a reasonable amount.
Re:Doesn't pay enough (Score:2, Insightful)
Re:Doesn't pay enough (Score:2)
Re:Doesn't pay enough (Score:2)
Ah, but I think you miss the point. How much to you get paid for a couple minutes at your regular job? Add 65 cents to that, and that's how much you'll get paid at your normal job to slack off and surf Amazon.
Re:Doesn't pay enough (Score:3, Insightful)
Re:Doesn't pay enough (Score:2)
B
Re:Doesn't pay enough (Score:2)
Market based? (Score:2)
Review??? or Multiple Tests/Hits (Score:2)
How many "agreeing" hits would you need before you can "accept" that the result is valid?
Elbonia (Score:3, Funny)
Re:Doesn't pay enough (Score:2)
Re:Doesn't pay enough (Score:5, Insightful)
100 images / hour * $0.03 / image == $3.00 / hour.
So, you've just busted your ass to crank through one image every 36 seconds for a solid hour, and you have three dollars to show for it. There are definitely parts of the world where that would be a fantastic income, but my home country isn't one of them.
It depends (Score:3, Insightful)
If you're scoffing because you're already employed at a job that pays better, then you're doing what you should. Somebody already values your labor more presumably because you are more productive doing that job rather than identifying items in pictures. I work plenty of hours and am well compensated for it. My remaining time is very valuable to me. Perhaps one should not scoff, but politely say "No, thanks." The job one might scoff at today might be the job that saves your ass tomorrow.
On the other h
Re:Doesn't pay enough (Score:5, Funny)
Re:Doesn't pay enough (Score:3, Insightful)
Re:Doesn't pay enough (Score:3, Insightful)
Let me translate... If it were write a product description for the pictured computer parts and you were waiting for your kernel to compile while twiddling your thumbs, why not make 65 cents?
Re:Doesn't pay enough (Score:4, Informative)
So basically, if it takes you 5 minutes to write a brief product description, and you churn through them all day, you're making $7.80, which is better than minimum wage. Not a good proprosition if you're clueless about auto parts and have to research everything as described, but I don't think that's the intended optimal target for completing the task (although it someone's dumb enough to spend half an hour or more per description for a crappy hourly wage, they're more than welcome). The optimal target to take up that task is someone who already knows a lot about car parts. Chances are if you're an Autozone (auto parts store chain) employee, you could get most of the descriptions done in under 5 minutes with little to no side research, because you already have the domain-specific knowledge. That's the guy who will be drawn to answer that question.
So the key to making effective money at this scheme is to skip tasks that you don't think you're "better than average" at - kinda like the job marketplace in real life.
Re:Doesn't pay enough (Score:4, Informative)
What about federal labor laws? (Score:2)
Re:What about federal labor laws? (Score:3, Informative)
Re:Doesn't pay enough (Score:2)
I see nothing to indicate that only us wealthy gringos can participate. I haven’t yet checked out the payment end of it, but I would assume that it will eventually be a worldwide thing, making it a nice, level, globalized playing field.
Whether that’s a good thing or a bad thing is left as an exercise to the reader.
$/hr (Score:3, Interesting)
Re:$/hr (Score:5, Informative)
There is an API, maybe if someone made a page that just displays the images and sends in the result when you click on the image instead of having to click twice for each HIT, you could go faster and make much more money.
Re:$/hr (Score:2)
If it takes you more than 10 or 15 seconds from the instant you read the description for a hit to the instant you are available to read the description for the next hit after successfully completing the first one, it's not really worth your time (at least, IMHO).
3 cents for 1 hours work? (Score:5, Funny)
Sounds interesting but.... (Score:5, Insightful)
Re:Sounds interesting but.... (Score:5, Informative)
Well, mturk.amazon.com [amazon.com] redirects you to www.mturk.com [mturk.com]... seems to imply something.
Re:Sounds interesting but.... (Score:5, Informative)
The whois info looks a bit dodgy. I would have expected Amazon knew how to properly register domains...
Registrant:
MTAI, Inc.
P.O. Box 80626
Seattle, Washington 98108
United States
Registered through: GoDaddy.com
Domain Name: MTURK.COM
Created on: 22-Oct-01
Expires on: 22-Oct-06
Last Updated on: 11-Oct-05
Administrative Contact:
Hostmaster, MTAI mechturk1@hotmail.com
MTAI, Inc.
P.O. Box 80626
Seattle, Washington 98108
United States
2065794562 Fax --
Technical Contact:
Hostmaster, MTAI mechturk1@hotmail.com
MTAI, Inc.
P.O. Box 80626
Seattle, Washington 98108
United States
2065794562 Fax --
Re:Sounds interesting but.... (Score:2)
Re:Sounds interesting but.... (Score:4, Informative)
But in case this is some kind of phishing, they at least manipulated some reverse
records too
traceroute to www.mturk.com (207.171.166.182), 30 hops max, 40 byte packets
[...]
11 amazon-above.mpr1.iad5.us.mfnx.net.175.185.208.in
12 72.21.201.27 97.109 ms 97.347 ms 98.164 ms
13 166-182.amazon.com (207.171.166.182) 98.107 ms 97.069 ms 97.510 ms
Re:Sounds interesting but.... (Score:3, Informative)
whois 207.171.166.182
[Querying whois.arin.net]
[whois.arin.net]
OrgName: Amazon.com, Inc.
OrgID: AMAZON-4
Address: 605 5th Ave S
City: SEATTLE
StateProv: WA
PostalCode: 98104
Country: US
NetRange: 207.171.160.0 - 207.171.191.255
CIDR: 207.171.160.0/19
NetName: AMAZON-01
NetHandle: NET-207-171-160-0-1
Parent: NET-207-0-0-0-0
NetType: Direct Assignment
NameServer: NS-1.AMAZON.COM
NameServer:
Re:Sounds interesting but.... (Score:5, Informative)
Re:Sounds interesting but.... (Score:2, Informative)
Re:Sounds interesting but.... (Score:3, Informative)
Re:Sounds interesting but.... (Score:2, Informative)
another redirect (Score:2)
It seems like the company does investor relations,
hence the corporate-ir (investor relations)
I wouldn't worry too much about this, if its good enough for EarthLink [corporate-ir.net] its good enough for me
btw- how did you convince
same link but
http://phx.corporate-ir.net/phoenix.zhtml?c=176060 &p=irol-InfoReq [corporate-ir.net]
How long until some sick slashdotter posts the ... (Score:5, Funny)
The Future of Surveillance (Score:3, Insightful)
Crime in your neighborhood?
Get a webcam...
Re:The Future of Surveillance (Score:3, Interesting)
Lets say I've got a herd of cows in a remote location. I setup a few webcams. I put tracking anklets on the cows. If a tracker shows a cow leaving the fenceline, or malfunctions, or is tampered with, the webcams come on. Some random person on the internet gets the task of "count the cows, identify any people". In fact, two people get that task, for redundancy. They can pan and zoom and get a bonus for finding trouble. The whole thing could be run by
Contracting work worth big bucks (Score:5, Insightful)
After a quick review of the available tasks, I must say this looks like a huge scam. Most of the tasks are marketing oriented (e.g. copywriting, photo manipulation), for which experienced contractors get paid $30 to $50 per hour.
Only 75 cents to research and write a complete automotive product description? Are they kidding? Sure, they say I can copy the description from the manufacturer's Web site, but my time is still worth more than that. Besides, I think it's the responsibility of the manufacturer to make sure their Amazon listing is correct. That's how they do it on IMDB.
I can only hope the program will make more sense as they add more requesters and more tasks.
Re:Contracting work worth big bucks (Score:2)
Which, by the way, is owned by Amazon.
Re:Contracting work worth big bucks (Score:3, Interesting)
Most of the ones I saw were trivial tasks. Even the auto description was edit the auto description until it was human readable. Since they are trivial, people get bored doing them. The common solution has been to over-pay someone to do them, and have the pay offset their boredom. This interface provides a new idea: let people do them until they get bored, and pay them by the piece.
If your time is truly worth more, don't do them. But there are people who will find it an interesting diver
Good idea but for work required it should pay more (Score:4, Insightful)
Not only will the work most likely be shoddy, but it seems like they are trying to replace someone else's job by using this cheap online service.
Yes, for some it may provide rewards but if you calculate the amount of time spent on each item VS. the payment reward (usually a few pennies) it is just not worth someone's free time.
Why don't they just hire a staff of people to work on these 'HITS'?
Willing to compete with the Indians? (Score:5, Insightful)
none of the above (Score:2)
for example, the storefront that is being asked about might be cut off completely, showing instead the entirety of the neighboring store. This might be the "best" photo, however, it would NOT be what the requesting storefront would be looking for. If people consistently chose the nicest photo as opposed to the one that most closely matches the request, i think this s
Re:none of the above (Score:2)
Profit? (Score:5, Interesting)
1) Use the API to find a HIT, and sign up to complete it.
2) Create a new HIT that basically asks someone to complete the first HIT,
only for $0.01 less than the original HIT was offering.
3) Do this for every existing HIT.
4) Profit?
What is your time worth? (Score:4, Insightful)
So to accomplish the 3 cent task and make your time worth it, you should spend no more than about 2 and a half seconds from the second you begin to the second you finish and get approval.
On some of the higher paying ones, oh, say $0.40 for writing a full product review, you could devote almost a full minute!
Re:What is your time worth? (Score:3, Interesting)
Well, Google says $65000 per year = $0.123586182 per minute [google.com].
For people "worth" $65,000/year there are three cases:
Do A9's Dirty Work For Them (Score:2, Informative)
Reverse auction (Score:2)
Philip K. Dick (Score:5, Interesting)
Quick -- someone patent that storyline and sue Amazon for infringement!
Re:Philip K. Dick (Score:2)
Re:Philip K. Dick (Score:5, Informative)
Japanese manuals? (Score:4, Funny)
Keep them coming, Amazon!
Learning from experience (Score:2)
This article has recently been linked from Slashdot [slashdot.org].
Please keep an eye on the page history [wikipedia.org] for errors or vandalism.
Highly suspectful site. Do NOT give any detail (Score:4, Interesting)
It's registered through Godaddy.com, one of the companies spammers/phishers love to use.
It has hotmail contact addresses in whois. Impossible for a company like Amazon
No clue of such thing on official Amazon press room
http://phx.corporate-ir.net/phoenix.zhtml?c=17606
So if it looks like,acts like,runs like (amazon gigantic server farm slashdotted?) a regular phishing site, it is. Even if it made to Slashdot. I'd say pull the story until Amazon comes up with an explanation. Before any harm done.
It could be even a more "elite" hack including subdomain/DNS hacking. I am a spamcop mail customer and I see amazing things everyday.
In risk of looking very funny if it is not anything above, happily posting it.
Re:Highly suspectful site. Do NOT give any detail (Score:2, Informative)
Re:Highly suspectful site. Do NOT give any detail (Score:2)
Re:Highly suspectful site. Do NOT give any detail (Score:4, Informative)
Anyhow, I tried it. It recognised me and new my address. It's Amazon alright.
Netcraft confirms it, am I stupid or amazon.com? (Score:3, Insightful)
http://toolbar.netcraft.com/site_report?url=http:/
It is indeed in Amazon netblock but registering it through godaddy.com with a hotmail address... Gee, I wish I could show like 40 phishing mails I received with the same pattern.
Sadly there are many victims of phishing sites, and they get slashdotted because the database software can't handle that many requests.
I have never seen such a unserious whois from a big company like amazon. There are many registr
Re:Highly suspectful site. Do NOT give any detail (Score:3, Informative)
The link ultimately goes to:
http://www.amazon.com/gp/browse.html/104-7108086-1 879910?node=15879911 [amazon.com]
Which has links back to www.mturk.com
Looks legitimate, unless someone has really managed to pull one over on Amazon (and if so, why put it on its own domain?)
Legit site. Ignore idiots. (Score:3, Informative)
To verify the legitimacy of the site, manually type "amazon.com" into your browser's location bar, and hover over the "See all 32 Product Categories" tab. When it pops up the list, click "Web Services" and read the first item listed on that page, which is a press release announcing Amazon Mechanical Turk.
For extra points, do this only a machine which has been booted from a liveCD with DNS utilities and hosts file that you have personally audited.
Or just, you know, look at the fact that the Turk will, by
As McEnroe said, "You can not be serious"... (Score:2, Funny)
Or as I say today, "40 cents for a product description!?!?! Fuck off!"
Agent Smith, is that you? (Score:5, Funny)
Don't say I didn't warn you.
Re:Agent Smith, is that you? (Score:2)
Repent! The Singularity is upon us!
Or something like that... Truth be told I could do my job from home if the technological infrastructure could support.
Enough with job interviews that get your ideas... (Score:2)
So, they get a lot of results and they do not approve anyone of them because the quality is not enough. And they get the job done uh?
Yeah, sure... where do I sign... *sigh*
Sweatshop of the future? (Score:2, Interesting)
automating this (Score:5, Interesting)
For the image ones, couldn't you create 5 bots each with a different account and each one picks a different image and one picks None of these? One of them would be approved and you'd get paid, right?
Also if they are having humans approve your image selection before you get paid, isn't that as much effort as you making your original choice?
Alternative to RentACoder for non-coding tasks. (Score:2)
Nirut Test (Score:2)
A rose by any other name... (Score:2)
I think I get it... (Score:3, Funny)
How to avoid paying out: (Score:2, Funny)
We slashdotted Amazon? (Score:3, Funny)
I wonder if serving web pages counts as such a task. Because their computers sure are doing a crappy job of it at the moment.
Slahdot should use this.. (Score:4, Funny)
Employee or Lab Rat. . ? (Score:3, Interesting)
It reminds me of a little semi-scam some company had going in my town a few years back. . .
"You are invited to participate in a screen test of a new television series!"
People would go down and be a test-audience for a television pilot, and then fill out a questionnaire at the end. People, loving their TV culture, were tickled pink to be asked to do this. --Heck, they were even paid something like $15 for their participation!
So, a buddy of mine went to see what it was all about. . .
Basically, some marketing research firm had acquired the rights to an old pilot which never made it to air. They played this for people, and also played a bunch of adverts during the commercial breaks. The questionnaire asked a few boring questions about the pilot, but it also asked a curiously high number of questions about the ad spots. Stuff like, "Which of the two detergent packages in the ad did you find more appealing? The Blue or the Red?"
--Obviously the whole contrivance was designed to test market, uh, marketing.
Either way, by friend was amazed that nobody else seemed to catch on, took his fifteen bucks, and left shaking his head.
-FL
Message from Amazon Mechanical Turk Team (Score:5, Informative)
Collaborative Human Interpreter (CHI) (Score:3, Informative)
The idea is to harness this kind of thing to develop software for the global brain. [communitywiki.org]
CHI, a Collaborative Human Interpreter (Score:3, Insightful)
http://blog.outer-court.com/archive/2005-03-25-n4
While most of the comments here seem to focus on the Worker side of this (those getting paid for answering simple questions), there's also the Requester side -- programmers tapping into the power of "fake" (but working!) AI. (Ladies and gentleman, we present you the global brain... it can think for you if you micro-pay!) I think we can implement many new programs/ websites in completely new ways, and there may even be fresh commercial niche programs coming out of this. Maybe in 50 years, we'll include AMT (or similar services) into our software as naturally as we now include, say, SQL.
I wish the site was working better at the moment (even before it has been Slashdotted, it was behaving strangely), and I wish it wouldn't ask me for a US bank account (being from Germany, that kinda hinders me from working with it).
Re:Will the real mechanical turk please stand up.. (Score:2)
Re:Lousy payscale. (Score:2)
It's legit (Score:2, Informative)