Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×
The Internet Businesses Software

Amazon's Mechanical Turk 375

rscoggin writes "Amazon.com has a new program that wants you to 'Complete simple tasks that people do better than computers. And, get paid for it.' (example: 'Is there a pizza parlour in this photograph?'). For each task you complete you get a small payment, usually ranging from a few cents to a little under a dollar. It's named the Amazon Mechanical Turk after a famous hoax from the 19th century. Kill time and get paid in tiny increments to boot!" Similar to Google Answers, there seems to be a reliability ratings system and some incentives.
This discussion has been archived. No new comments can be posted.

Amazon's Mechanical Turk

Comments Filter:
  • Great... (Score:5, Funny)

    by TubeSteak ( 669689 ) on Friday November 04, 2005 @08:52AM (#13949400) Journal
    Great... Another way for /.'ers to waste time at work.
    GOOD JOB AMAZON
  • by mrseigen ( 518390 ) on Friday November 04, 2005 @08:53AM (#13949406) Homepage Journal
    This could get addictive.
    • Actually, you gave me an interesting idea.

      "Micropayments" are used a lot of send small payments to 'charitable' organizations--not just bona fide charities, but things like Open Source projects.

      Suppose you could sign up and do this, but have the proceeds sent to charity? Getting 3 cents isn't at all useful for me. If many people sent their 3 cents to a developing country, it would matter.

      I can't navigate the site that well right now (everything's returning errors), so I'm not sure if this is in the works or
  • CAPTCHAs (Score:3, Interesting)

    by CableModemSniper ( 556285 ) <.moc.liamg. .ta. .odlapacnagol.> on Friday November 04, 2005 @08:53AM (#13949410) Homepage Journal
    Does this mean I can get paid for breaking CAPTCHAs?
    • Re:CAPTCHAs (Score:5, Informative)

      by Agelmar ( 205181 ) * on Friday November 04, 2005 @08:57AM (#13949427)
      Actually, yes. The whole motivation from this came from the same person who invented the CAPTCHA, and was explained in his thesis defense on Wednesday. Abstract for those who care:

      Subject: Thesis Oral - Luis von Ahn

      November 2, 2005
      Luis von Ahn
      12:00 PM, 3305 Newell-Simon Hall
      Thesis Oral
      Title: Human Computation

      Abstract:

      Tasks like image recognition are trivial for humans, but continue to
      challenge even the most sophisticated computer programs. This thesis
      introduces a paradigm for utilizing human processing power to solve
      problems that computers cannot yet solve. Traditional approaches to
      solving such problems focus on improving software. I advocate a novel
      approach: constructively channel human brainpower using computer games.
      For example, the ESP Game, introduced in this thesis, is an enjoyable
      online game -- many people play over 40 hours a week -- and when people
      play, they help label images on the Web with descriptive keywords. These
      keywords can be used to significantly improve the accuracy of image
      search. People play the game not because they want to help, but because
      they enjoy it.

      I introduce three other examples of games with a purpose: Peekaboom,
      which helps determine the location of objects in images, Phetch, which
      collects paragraph descriptions of arbitrary images to help
      accessibility of the Web, and Verbosity, which collects common-sense
      knowledge. I also show that, in principle, every problem that could be
      solved by a computer, today or in the future, could be solved using
      enjoyable computer games.

      In addition, I introduce CAPTCHAs, automated tests that humans can pass
      but computer programs cannot. CAPTCHAs take advantage of human
      processing power in order to differentiate humans from computers, an
      ability that has important applications in practice.

      The results of this thesis are currently in use by hundreds of Web sites
      and companies around the world, and some of the games presented here
      have been played by over 100,000 people. Practical applications of this
      work include improvements in problems such as: image search,
      adult-content filtering, spam, common-sense reasoning, computer vision,
      accessibility, and security in general.

      Thesis Committee:
      Manuel Blum, Chair
      Takeo Kanade
      Michael Reiter
      Josh Benaloh, Microsoft Research
      Jitendra Malik, University of California, Berkeley
      • Why not make a program that captures the CAPTCHA image on slashdot and uploads it on a webserver for people to read it and enter the text and then that information is sent back to a robot trying to post and then voila!
      • Re:CAPTCHAs (Score:3, Funny)

        by glinden ( 56181 ) *
        The whole motivation from this came from the same person who invented the CAPTCHA, and was explained in his thesis defense on Wednesday.
        When you say this thesis was the motivation for Amazon Mechanical Turk, what exactly do you mean? Luis von Ahn isn't at Amazon, is he?
  • by Anonymous Coward on Friday November 04, 2005 @08:54AM (#13949415)
    - can you see boobs in the picture ?
    - Is there a donkey in the picture ?
    - Can you see the can of whipped cream ?
    - is there chocolate paint involved..

    Advanced indexing of Pr0n, humanity is moving forward, no doubt.
  • by RandoX ( 828285 ) on Friday November 04, 2005 @08:59AM (#13949436)
    Pepsi pays Amazon 3 cents for product placement. You are shown an image of a Pepsi can. "What kind of soda is this?" "pepsi", you answer. You get paid 2 cents.
  • by Dekortage ( 697532 ) on Friday November 04, 2005 @08:59AM (#13949439) Homepage

    According to this earlier Slashdot report [slashdot.org], the spam industry has been doing this for awhile with free porn.

    I'm curious to know if Amazon is going to use the cumulative results to try to "train" computers, or if it really is just for the money. The requirements include being over 18, so you can't pimp your kids to click through this stuff for cash (though I'm sure it will happen).

  • Doesn't pay enough (Score:5, Insightful)

    by slavemowgli ( 585321 ) on Friday November 04, 2005 @08:59AM (#13949440) Homepage

    Maybe it's just me, but it seems that it's not really worth it. Consider the following task, for example:

    Your task is to create a new product description for a product in the Amazon.com Automotive catalogue. The Product Description provides an additional opportunity to tell the customer about the product. This HIT will require some product research to complete. Approval depends on following the instructions and the quality of your submission, determined by a manual review.

    Guess how much you get paid for that. 2 dollars? 3? That wouldn't be unreasonable, I think, considering that you're supposed to write an entire product description from scratch for which additional "research" is required. The actual amount paid is only 65 cents, though.

    Maybe it's just me, but if I check to see how much I need to work in my regular job to make 65 cents, then it does not make any sense to invest more than a few minutes into a task like this, and it seems that it would take more than that to actually complete it. The fact that there's a review required afterwards doesn't exactly make things better, either - if what you did gets rejected, then you've essentially worked for nothing (I wonder if there's anything that keeps amazon from still using your description in this case, too...).

    In other words, the whole thing seems like a good idea in theory, but it won't really take off until they're willing to actually pay you a reasonable amount.

    • by RandoX ( 828285 )
      Just do it AT work. Get paid twice.
    • That was my thought as well. "Write a product description, research required" seems to me that .65 (or less in some cases) is... piddly?
    • Maybe it's just me, but if I check to see how much I need to work in my regular job to make 65 cents, then it does not make any sense to invest more than a few minutes into a task like this, and it seems that it would take more than that to actually complete it.

      Ah, but I think you miss the point. How much to you get paid for a couple minutes at your regular job? Add 65 cents to that, and that's how much you'll get paid at your normal job to slack off and surf Amazon.

    • by iphayd ( 170761 )
      Not to mention that this is the type of thing a copywriter at a marketing firm gets big bucks to do, so essentially you are doing their job for free for Amazon.
    • I agree. The jobs that really are trivial pay nothing and the ones that pay aren't at all trivial. 65 cents or 75 cents isn't nearly enough for what they are asking.
    • I don't know if all of the tasks are going to be from Amazon. It looks to me like anyone can submit "work" through the API, and the market will decide what gets done and what does not. If that 65 cent job sits long enough, and Amazon wants to get it done, they'll pony up more cash until someone thinks it's worthwhile.
    • If a person has to review each and every hit, then that person could have done the work themselves.

      How many "agreeing" hits would you need before you can "accept" that the result is valid?

    • Elbonia (Score:3, Funny)

      by HermanAB ( 661181 )
      Well, that is probably a whole month's average income for someone in Elbonia...
    • If you notice most are "find the best picture" for $0.03. Of course the site rules really slow right now so its not worth your time, but if it ran faster it would be something to do if you could crank through a hundred an hour. Yes, the write descriptions are stupid, though I think correct this grammer might work..
      • by egomaniac ( 105476 ) on Friday November 04, 2005 @10:09AM (#13949950) Homepage
        If you notice most are "find the best picture" for $0.03. Of course the site rules really slow right now so its not worth your time, but if it ran faster it would be something to do if you could crank through a hundred an hour.

        100 images / hour * $0.03 / image == $3.00 / hour.

        So, you've just busted your ass to crank through one image every 36 seconds for a solid hour, and you have three dollars to show for it. There are definitely parts of the world where that would be a fantastic income, but my home country isn't one of them.
    • by Anne_Nonymous ( 313852 ) on Friday November 04, 2005 @10:06AM (#13949935) Homepage Journal
      It's not about the money; it's about fucking with Amazon, by giving hilariously incorrect answers.
      • Bad accounts are easy to detect. This system could easily be made quite fraud-proof.
        1. Tie accounts to something valuable, like Amazon accounts with a verified purchase history, to prevent mass signps. Easy.
        2. Randomly check answers by randomly posing the same question to two different accounts. This step is infinitely tuneable; more checks means more fraud resistance at a higher overhead cost. New accounts and accounts with suspicious activity could be checked much more often. Reliable accounts could b
    • That job would never be worth 65 cents to me, but if I were a mechanic and the additional research could be labelled as my common knowledge, and I was IM'ing with my girlfriend at midnight and she was a slow typer...

      Let me translate... If it were write a product description for the pictured computer parts and you were waiting for your kernel to compile while twiddling your thumbs, why not make 65 cents?
    • by photon317 ( 208409 ) on Friday November 04, 2005 @10:48AM (#13950333)

      So basically, if it takes you 5 minutes to write a brief product description, and you churn through them all day, you're making $7.80, which is better than minimum wage. Not a good proprosition if you're clueless about auto parts and have to research everything as described, but I don't think that's the intended optimal target for completing the task (although it someone's dumb enough to spend half an hour or more per description for a crappy hourly wage, they're more than welcome). The optimal target to take up that task is someone who already knows a lot about car parts. Chances are if you're an Autozone (auto parts store chain) employee, you could get most of the descriptions done in under 5 minutes with little to no side research, because you already have the domain-specific knowledge. That's the guy who will be drawn to answer that question.

      So the key to making effective money at this scheme is to skip tasks that you don't think you're "better than average" at - kinda like the job marketplace in real life.
    • by TheSync ( 5291 ) on Friday November 04, 2005 @11:32AM (#13950809) Journal
      One billion people [econotarian.org] on planet earth make less than $1 per day. Perhaps you are not the target audience...
  • $/hr (Score:3, Interesting)

    by MyLongNickName ( 822545 ) on Friday November 04, 2005 @08:59AM (#13949441) Journal
    Anyone want to make an estimation of $/hr earned doing this? I'm at work, and don't have the balls to spend 20 minutes earning cash online ;)
    • Re:$/hr (Score:5, Informative)

      by yellowbkpk ( 890493 ) * on Friday November 04, 2005 @09:04AM (#13949474)
      I did nine 3-cent HITs in about 5 or 6 minutes, so that's about 3.25/hour. The lag for me was in waiting for the images to download and clicking on the "Accept HIT" button repeatedly.

      There is an API, maybe if someone made a page that just displays the images and sends in the result when you click on the image instead of having to click twice for each HIT, you could go faster and make much more money.
    • At $0.03 cents/hit, you'll want to aim for around 5 sec/hit to earn about $20/hr.

      If it takes you more than 10 or 15 seconds from the instant you read the description for a hit to the instant you are available to read the description for the next hit after successfully completing the first one, it's not really worth your time (at least, IMHO).

  • by mustafap ( 452510 ) on Friday November 04, 2005 @08:59AM (#13949442) Homepage
    Wow, I can give up my day job!
  • by antifoidulus ( 807088 ) on Friday November 04, 2005 @09:00AM (#13949445) Homepage Journal
    I'm a bit too paranoid to type my Amazon user name and password into a site that isn't on the main amazon.com domain....I can't find it mentioned anywhere on amazon's main site. Can somebody a little bit less of a wuss tell me if it is legit?
  • by Anonymous Coward on Friday November 04, 2005 @09:02AM (#13949459)
    following HIT: "Is there a goat in that picture?"
  • by wombatmobile ( 623057 ) on Friday November 04, 2005 @09:04AM (#13949473)

    Crime in your neighborhood?

    Get a webcam...

    • That's a good idea. I would actually pay for that.

      Lets say I've got a herd of cows in a remote location. I setup a few webcams. I put tracking anklets on the cows. If a tracker shows a cow leaving the fenceline, or malfunctions, or is tampered with, the webcams come on. Some random person on the internet gets the task of "count the cows, identify any people". In fact, two people get that task, for redundancy. They can pan and zoom and get a bonus for finding trouble. The whole thing could be run by
  • by Silverhammer ( 13644 ) on Friday November 04, 2005 @09:05AM (#13949480)

    After a quick review of the available tasks, I must say this looks like a huge scam. Most of the tasks are marketing oriented (e.g. copywriting, photo manipulation), for which experienced contractors get paid $30 to $50 per hour.

    Only 75 cents to research and write a complete automotive product description? Are they kidding? Sure, they say I can copy the description from the manufacturer's Web site, but my time is still worth more than that. Besides, I think it's the responsibility of the manufacturer to make sure their Amazon listing is correct. That's how they do it on IMDB.

    I can only hope the program will make more sense as they add more requesters and more tasks.

    • I think it's the responsibility of the manufacturer to make sure their Amazon listing is correct. That's how they do it on IMDB.

      Which, by the way, is owned by Amazon.

    • So, don't do it.

      Most of the ones I saw were trivial tasks. Even the auto description was edit the auto description until it was human readable. Since they are trivial, people get bored doing them. The common solution has been to over-pay someone to do them, and have the pay offset their boredom. This interface provides a new idea: let people do them until they get bored, and pay them by the piece.

      If your time is truly worth more, don't do them. But there are people who will find it an interesting diver
  • by digitaldc ( 879047 ) on Friday November 04, 2005 @09:10AM (#13949499)
    They are asking you to rewrite product descriptions and will pay you 60 cents?
    Not only will the work most likely be shoddy, but it seems like they are trying to replace someone else's job by using this cheap online service.
    Yes, for some it may provide rewards but if you calculate the amount of time spent on each item VS. the payment reward (usually a few pennies) it is just not worth someone's free time.
    Why don't they just hire a staff of people to work on these 'HITS'?
  • by nysus ( 162232 ) on Friday November 04, 2005 @09:10AM (#13949500)
    I'm interested to know if those living overseas can participate. If so, they would drive down the labor costs so much that only truly desperate Americans would participate in this piecework scheme.
  • i'm finding a bunch of these storefront pictures as being horrible pictures... perhaps they're testing the reliability of the system here?

    for example, the storefront that is being asked about might be cut off completely, showing instead the entirety of the neighboring store. This might be the "best" photo, however, it would NOT be what the requesting storefront would be looking for. If people consistently chose the nicest photo as opposed to the one that most closely matches the request, i think this s

    • They're not asking for a photography judge here, they're asking for you to pick the best image of the storefront in question. If the storefront isn't there, then you choose "None of the Above" and move on to the next one.
  • Profit? (Score:5, Interesting)

    by DevolvingSpud ( 774770 ) on Friday November 04, 2005 @09:15AM (#13949534)
    Since this is all web-services driven, it seems to me you could create an interesting cycle with a simple program:

    1) Use the API to find a HIT, and sign up to complete it.
    2) Create a new HIT that basically asks someone to complete the first HIT,
          only for $0.01 less than the original HIT was offering.
    3) Do this for every existing HIT.
    4) Profit?
  • by shoolz ( 752000 ) on Friday November 04, 2005 @09:17AM (#13949548) Homepage
    Rather than think about how much you could make per hour on this, think about how much your time is worth. Are you worth $65,000 per year? Maybe you're worth more or you value your time more? In any case, at $65,000 per year, you make about $0.52 per minute.

    So to accomplish the 3 cent task and make your time worth it, you should spend no more than about 2 and a half seconds from the second you begin to the second you finish and get approval.

    On some of the higher paying ones, oh, say $0.40 for writing a full product review, you could devote almost a full minute!
    • Are you worth $65,000 per year? Maybe you're worth more or you value your time more? In any case, at $65,000 per year, you make about $0.52 per minute.

      Well, Google says $65000 per year = $0.123586182 per minute [google.com].

      For people "worth" $65,000/year there are three cases:
      1. You are on salary at a day-job ($65,000 / year, irrespective of what you're doing as long as you don't get fired - commute, come in one saturday to meet a deadline, read slashdot instead of working, slip out early, come in late, or come in
  • Acutally, all the tasks that I saw involved processing data for A9's block-level search and "tour". [wired.com] Seems like a clever, cheap way to organize the insane amount of data they have mapped for this project.
  • Increase the value of the job till it's worth someone's while to actually do it. Everyone benefits. Any rejected work remains owned by author.
  • Philip K. Dick (Score:5, Interesting)

    by baxissimo ( 135512 ) on Friday November 04, 2005 @09:34AM (#13949651)
    The reminds me of the Philip K. Dick novel in which the main character thinks he lives an ordinary life, and who solves the daily puzzle in the newspaper every day for cheap entertainment. In reality, though, the whole town he lives in is a front, and the fun puzzles he's solving in the newspaper are actually cleverly disguised military strategy problems of some sort.

    Quick -- someone patent that storyline and sue Amazon for infringement!
  • by Crouty ( 912387 ) on Friday November 04, 2005 @09:35AM (#13949667)
    Wow, I can't wait to unleash my grammar nazi skills on the automated translations of japanese manuals of electronic devices!

    Keep them coming, Amazon!

  • Very smart...someone is keeping tabs on these things. At the top of the Wikipedea page:

    This article has recently been linked from Slashdot [slashdot.org].
    Please keep an eye on the page history [wikipedia.org] for errors or vandalism.

  • by Ilgaz ( 86384 ) on Friday November 04, 2005 @09:37AM (#13949676) Homepage
    Guys, let me tell.

    It's registered through Godaddy.com, one of the companies spammers/phishers love to use.

    It has hotmail contact addresses in whois. Impossible for a company like Amazon

    No clue of such thing on official Amazon press room
    http://phx.corporate-ir.net/phoenix.zhtml?c=176060 &p=irol-news [corporate-ir.net]

    So if it looks like,acts like,runs like (amazon gigantic server farm slashdotted?) a regular phishing site, it is. Even if it made to Slashdot. I'd say pull the story until Amazon comes up with an explanation. Before any harm done.

    It could be even a more "elite" hack including subdomain/DNS hacking. I am a spamcop mail customer and I see amazing things everyday.

    In risk of looking very funny if it is not anything above, happily posting it.

  • or as we used to say in the UK, "Pull the other one, it's get bells on"

    Or as I say today, "40 cents for a product description!?!?! Fuck off!"

  • by OldManAndTheC++ ( 723450 ) on Friday November 04, 2005 @09:40AM (#13949706)
    I hope you all can see where this is going:

    • 2005: Amazon introduces Mechanical Turk program. Thousands of underpaid geeks sign up and start clicking mindlessly.
    • 2010: Home catheterization and feeding tube kit eliminates need to leave workstation. Productivity skyrockets.
    • 2015: Direct neural interface improves response times one million-fold.
    • 2020: The Matrix

    Don't say I didn't warn you.

  • Your task is to edit an existing Automotive product title to make it more human readable and update and add additional feature points about the product. This HIT will require some research to complete. Approval depends on the quality of your title and feature points, determined by a manual review.

    So, they get a lot of results and they do not approve anyone of them because the quality is not enough. And they get the job done uh?

    Yeah, sure... where do I sign... *sigh*
  • Are the 3rd world drones that will do this pulling themselves up by their bootstraps into the information age or is this some kind of futuristic Dickensian sweatshop where piecemeal work is paid at three cents a click?
  • automating this (Score:5, Interesting)

    by mboedick ( 543717 ) on Friday November 04, 2005 @09:52AM (#13949821)

    For the image ones, couldn't you create 5 bots each with a different account and each one picks a different image and one picks None of these? One of them would be approved and you'd get paid, right?

    Also if they are having humans approve your image selection before you get paid, isn't that as much effort as you making your original choice?

  • A lot of very small copywriting and image manipulation tasks crop up on RAC. If MTurk has lower transaction fees then I expect to see most of them moving over, providing a very nice user base for the service.
  • Hmm... I wonder how long it will be until these people can pass the Nirut Test:

    The Nirut test is a proposal for a test of a human being's capability to perform machine-like computation. Described by Nala Nirut in the 1993 paper "Computing and human intelligence", it proceeds as follows: a human mathematician engages in a conversation with two other parties, one a human and the other a machine; if the judge cannot reliably tell which is which, then the human subject is said to pass the test. In order to k

  • So basically it’s Rent-A-Coder for stoners, right?

  • by 93 Escort Wagon ( 326346 ) on Friday November 04, 2005 @10:04AM (#13949916)
    ... basically it's just like a sneaker factory in Vietnam, but with a cool-sounding name.
  • Make the site dog ass slow so you can ID 3 photos in 10 minutes. I'm up to 9 cents! I guess I should get back to coding, since thats what the OTHER people are paying me to do right now...
  • by Dachannien ( 617929 ) on Friday November 04, 2005 @10:24AM (#13950095)
    Complete simple tasks that people do better than computers.

    I wonder if serving web pages counts as such a task. Because their computers sure are doing a crappy job of it at the moment.

  • by objekt ( 232270 ) on Friday November 04, 2005 @10:56AM (#13950431) Homepage
    ..they could ask us questions like "Is this a dupe?"

  • by Fantastic Lad ( 198284 ) on Friday November 04, 2005 @10:58AM (#13950451)
    While this Amazon thing might be a genuine attempt to farm out real work to people for chump-change, with the site Slashdotted, I can only sit here and wonder. . .

    It reminds me of a little semi-scam some company had going in my town a few years back. . .

    "You are invited to participate in a screen test of a new television series!"

    People would go down and be a test-audience for a television pilot, and then fill out a questionnaire at the end. People, loving their TV culture, were tickled pink to be asked to do this. --Heck, they were even paid something like $15 for their participation!

    So, a buddy of mine went to see what it was all about. . .

    Basically, some marketing research firm had acquired the rights to an old pilot which never made it to air. They played this for people, and also played a bunch of adverts during the commercial breaks. The questionnaire asked a few boring questions about the pilot, but it also asked a curiously high number of questions about the ad spots. Stuff like, "Which of the two detergent packages in the ad did you find more appealing? The Blue or the Red?"

    --Obviously the whole contrivance was designed to test market, uh, marketing.

    Either way, by friend was amazed that nobody else seemed to catch on, took his fifteen bucks, and left shaking his head.


    -FL

  • by Amazon Mturk Team ( 928506 ) on Friday November 04, 2005 @12:45PM (#13951514)
    We're grateful to have been Slashdotted! Our beta site, mturk.amazon.com, is experiencing the Slashdot effect. You can still read about Amazon Mechanical Turk and its web services APIs at www.amazon.com/webservices. Also, send a blank email to aws@amazon.com if you want us to email you when page load times recover. The Amazon Mechanical Turk Team
  • by LionKimbro ( 200000 ) on Friday November 04, 2005 @01:26PM (#13951833) Homepage
    There was an interesting article a while back about a Collaborative Human Interpreter (CHI). [outer-court.com]

    The idea is to harness this kind of thing to develop software for the global brain. [communitywiki.org]
  • by j_philipp ( 803945 ) on Friday November 04, 2005 @06:32PM (#13954398) Homepage
    I like the Mechanical Turk service. It's just like my CHI proposal from half a year ago made real.
    http://blog.outer-court.com/archive/2005-03-25-n43 .html [outer-court.com]

    While most of the comments here seem to focus on the Worker side of this (those getting paid for answering simple questions), there's also the Requester side -- programmers tapping into the power of "fake" (but working!) AI. (Ladies and gentleman, we present you the global brain... it can think for you if you micro-pay!) I think we can implement many new programs/ websites in completely new ways, and there may even be fresh commercial niche programs coming out of this. Maybe in 50 years, we'll include AMT (or similar services) into our software as naturally as we now include, say, SQL.

    I wish the site was working better at the moment (even before it has been Slashdotted, it was behaving strangely), and I wish it wouldn't ask me for a US bank account (being from Germany, that kinda hinders me from working with it).

"The C Programming Language -- A language which combines the flexibility of assembly language with the power of assembly language."

Working...