Forgot your password?
typodupeerror
Software Technology Science

Casting Doubt On the Hawkeye Ball-Calling System 220

Posted by timothy
from the nudging-the-odds-means-smarter-bets dept.
Human judgment by referees is increasingly being supplemented (and sometimes overridden) by computerized observation systems. nuke-alwin writes "It is obvious that any model is only as accurate as the data in it, and technologies such as Hawkeye can never remove all doubt about the position of a ball. Wimbledon appears to accept the Hawkeye prediction as absolute, but researchers at Cardiff University will soon publish a paper disputing the accuracy of the system."
This discussion has been archived. No new comments can be posted.

Casting Doubt On the Hawkeye Ball-Calling System

Comments Filter:
  • Why not use... (Score:5, Insightful)

    by Kagura (843695) on Saturday June 28, 2008 @11:45PM (#23987315)

    Why not use a radio transmitter in the tennis ball (or soccer ball or whatever) to record its exact position? I am certain this has been discussed and I wouldn't be surprised if it's already in use. The article's "Hawkeye" just works by optical analysis.

    • If you leave the store parking lot, one of the wheels locks.

    • Re:Why not use... (Score:5, Informative)

      by Bun (34387) on Sunday June 29, 2008 @12:14AM (#23987461)

      Why not use a radio transmitter in the tennis ball (or soccer ball or whatever) to record its exact position? I am certain this has been discussed and I wouldn't be surprised if it's already in use. The article's "Hawkeye" just works by optical analysis.

      It's been tried in soccer. The latest attempts were prior to the last couple of World Cups IIRC, but the systems were plagued with problems, not the least of which was the survival of the transmitter.

      http://www.gizmag.com/go/2790/ [gizmag.com]

      • Re:Why not use... (Score:5, Interesting)

        by InadequateCamel (515839) on Sunday June 29, 2008 @12:31AM (#23987535)

        Further to that, if the transmitter can't survive in a soccer ball (where a well-struck shot probably moves around 120-130 kph) then there's no way it will handle travelling over 200 kph after a serve, followed by a (at least) 100 kph forehand return (a net >-300 kph in a fraction of a second!).

        Also, a radio transmitter cannot account for the distortion of a ball upon impact, which will depend on velocity, angle of rotation, angle of impact, surface being played on, etc etc etc...

        • by Nullav (1053766)

          Further to that, if the transmitter can't survive in a soccer ball (where a well-struck shot probably moves around 120-130 kph) then there's no way it will handle travelling over 200 kph after a serve, followed by a (at least) 100 kph forehand return (a net >-300 kph in a fraction of a second!).

          Red flag! (Oh god. Now I really want to see someone try that.)

        • Re: (Score:3, Insightful)

          by rant64 (1148751)

          Also, a radio transmitter cannot account for the distortion of a ball upon impact

          I seriously doubt that an umpire can.

          Hawkeye's also being used in snooker now, and it actually looks very accurate. The refs always re-spot the ball at least 2 inches away from the spot where it was, and I don't see why they're not using this more often.

          Honestly, even if the Hawkeye system is off by a few millimeters, if I were a pro tennis player then I'd rather have a call which is at most 3mm off than being called by an umpire who maybe wasn't paying close attention and calls whatever he thinks is righ

    • Re:Why not use... (Score:5, Interesting)

      by icegreentea (974342) on Sunday June 29, 2008 @12:24AM (#23987509)
      Assuming you could build a radio transmitter tough enough to handle it...

      With tennis balls, I imagine there would be problems with balance and the response of the ball. Especially with such a small ball, mounting a rugidtized radio transmitter (a ball probably has to go through 20gs or something) would probably mess with the balance and how the ball deforms. Not to mention, unless you can mount the system directly in the center of the ball, then you still have a margin of error the diameter of the ball. I imagine that would be fairly significant amount of error in tennis (perhaps on the same level as this Hawkeye system?) when calling the lines.
      • by Amouth (879122)

        wouldn't this be a good use for RFID tags/? they should be tough enough to handel it

        • Re:Why not use... (Score:5, Interesting)

          by jfim (1167051) on Sunday June 29, 2008 @12:59AM (#23987661)

          Triangulation of radio signals is not accurate enough to give sub-centimeter accuracy and the added mass to the tennis ball would probably cause the players to have some objection to adding a radio transmitter into the ball.

          The claim that the Hawkeye system gives an average of about four millimetres of error seems somewhat reasonable, given that we're getting accuracy greater better than two centimetres on detecting objects with a single camera with optics as large as the last segment of a typical pinky. (FWIW, here's a short demo [youtube.com] of what we're working on for our autonomous underwater vehicle [etsmtl.ca])

          However, the suggestion to display the error range for a particular shot and leaving the final decision to a human from TFA is quite reasonable and is how it should be. Blindingly trusting technology or discarding it altogether is unreasonable.

          • Re: (Score:2, Insightful)

            by Anonymous Coward

            ...leaving the final decision to a human from TFA is quite reasonable and is how it should be.

            No it isn't, it's ridiculous. On what basis is an umpire supposed to over-rule a machine with 4mm accuracy? True, the machine may be "wrong" from time to time but by trusting a machine you create a deterministic rule set which is completely neutral. It is precisely the fact of removing a human from the equation that makes Hawkeye so useful and so MacEnroe proof.

          • Re: (Score:3, Insightful)

            by nine-times (778537)

            Blindingly trusting technology or discarding it altogether is unreasonable.

            I disagree. Since this is a game, it seems to me the most important thing is that the rules are applied consistently and impartially. Accuracy may be the goal of making the rules, but once the rules are set, I'm much more concerned about the consistency and impartiality.

            They played tennis for quite a long time without the technology, and so it's evident that discarding it altogether wouldn't be so bad. Accuracy isn't really the issue. You could decide all disputes with the roll of a 12-sided die, and i

            • Re: (Score:3, Insightful)

              by CastrTroy (595695)
              They've been using human referees for a long time, but they really haven't had any other option up until this point. You could say the same thing about the introduction of the car. Everybody's been using horse and buggies for getting around for a long time. Since they've gone so long without the technology, it's evident that discarding it wouldn't be so bad. Even if the error is as large as 1 cm, I would say that's pretty good. How good is the accuracy of human vision at a distance of 40 feet with an o
            • Re: (Score:3, Informative)

              by bh_doc (930270)

              For example, if it were truly random, then players might start appealing every call. If they have nothing to lose and a random chance at success, then why not?

              Wimbeldon, IIRC, has a limit of 3 appeals. Just as an example.

    • Because that's not the issue. You'll always have uncertainty in systems. The study argues that the public perceives these systems as infallible, and therefore believe that technology can provide a final, absolute arbitration. The study is commenting on this tendency in lay people (i.e., people without specialized knowledge of the system), and warns against the corollaries that stem from such assumptions. Also, the title is bad: they are merely looking at the issue through the lens of Hawk-Eye, they're no
      • by aussie_a (778472)

        The issue isn't whether its infallible, but whether its more accurate then a human. If it is, then we should replace umpires with the Hawkeye systems. Will there be bad calls? Sure. But they will happen less.

    • Re:Why not use... (Score:5, Interesting)

      by Sethumme (1313479) on Sunday June 29, 2008 @01:38AM (#23987825)
      I still don't understand why there isn't more research on developing a surface for the out-of-bounds area that temporarily registers the exact impression of any impact on it.
      I envision something that looks like a big LCD touch screen (but more durable). Every time something made contact with the active surface, a record of the ball's "footprint" could be recorded (and even temporarily displayed wherever it touched the surface). That would allow for highly precise measurement of the ball's landing position, and it wouldn't need to incorporate any new materials into the ball itself. The active surface would only need to be in the out of bounds area, and even then, it would only need to be half a foot wide in order to cover the important zone where the ball's landing position is questionable.
      • by Anpheus (908711)

        We could call such a surface "Wet Paint" and use it whenever we need to determine if contact was made between the "Wet Paint" and another object.

      • by Don_dumb (927108) on Sunday June 29, 2008 @04:45AM (#23988545)
        At Wimbledon you could have chalk for the lines and if you were unsure if the ball hit the line one of the competitors could point out that

        "the chalk flew up"

      • Re: (Score:2, Funny)

        by Keychain (1249466)
        Yeah i hear they invented something like this : every time the ball touches the grounds it leaves a visible mark. i think they call it Complex Layer Against Yelling, but i doubt Wimbledon and its tradition are ready to take the plunge
    • Re:Why not use... (Score:4, Insightful)

      by Drathos (1092) on Sunday June 29, 2008 @01:47AM (#23987857)

      Fox tried to do that with hockey back in the 90s in order to make the puck easier to see on TV (personally, I've never had a problem seeing the puck). The Glow Puck was horrible. When there was a jam up in the corner, it would literally be bouncing all over the screen. It also changed the way the puck performed on the ice. Because of the electronics and battery inside, they couldn't freeze the puck like they normally do, causing it to bounce a lot more and not slide on the ice as easily.

      In a hollow sphere like a tennis ball, how would you keep the dynamics of the ball the same as they are when you add a transmitter to it? If you adhere it to the side, the ball will be off balance. If you create some internal structure/support to keep it centered, you change the deformation during a bounce/hit.

    • Re: (Score:3, Insightful)

      by jez9999 (618189)

      Could someone explain to me how this would be any more precise than high-quality optical analysis? Usually, in the slo-mo replay recorded by even the *average* quality cameras for TV audiences, you can almost always tell whether a ball was in or out. Make that higher quality cameras with a higher frame rate, and optical analysis seems like a very good way to do it, to me. It is, after all, what human umpires do.

  • The decision of which system to use: human, computer, human with computer check, computer with human check, committee vote, or what-not should be based on which has the lowest uncorrected error rate within limited time constraints.

    This assumes there is another method, such as post-analysis of videotape, that can find almost all uncorrected errors or at least give some good indication of the uncorrected error rate.

  • by Anonymous Coward on Saturday June 28, 2008 @11:46PM (#23987325)

    And ultra-accurate GPS like system that tracks the position of balls in nanosecond detail. They can call it Your Object Universal Remote Movement Observance Mechanism, or YOUR MOM for short.

  • Can this be applied to something useful. You know besides whether or not someone was out in a game of tennis?

    • by Joebert (946227)
      Yes. We will be able to determine once and for all whether or not they just grabbed my ass.
      • Re: (Score:2, Funny)

        by Anonymous Coward

        We will be able to determine once and for all whether or not they just grabbed my ass.

        You're a guy reading slashdot by yourself on a saturday night. It doesn't take any special technology to know the answer to that question.

    • Yea technology for sports is ridiculous. Better find new ways of killing other human beings more efficiently.
    • by InfoHighwayRoadkill (454730) on Sunday June 29, 2008 @03:49AM (#23988357) Homepage

      Yes, some people also want to use Hawkeye for some decisions in cricket, the sport that first used it. However the margin of error is far greater (approximately +- 2 inches) in cricket as the cameras have to be a lot further away due to the size of the pitch.

      Also Hawkeye finds it hard to pick up swinging, seaming and spinning balls. Basically anything that deviates off its theoretical trajectory either in the air or off the playing surface. Both of which are vital in the LBW decisions where the TV companies and doubtless the Hawkeye people would want to see it used.

      Obviously cricket is a far more useful game than tennis so does this answer your question?

      • by stranger_to_himself (1132241) on Sunday June 29, 2008 @04:46AM (#23988553) Journal

        Yes, some people also want to use Hawkeye for some decisions in cricket, the sport that first used it. However the margin of error is far greater (approximately +- 2 inches) in cricket as the cameras have to be a lot further away due to the size of the pitch.

        The other key difference in cricket is that Hawkeye is used to predict where the ball would have gone had it not hit a pad, whereas in tennis it only needs to say where the ball actually was.

      • Re: (Score:3, Informative)

        by Don_dumb (927108)
        The reason it isn't officially used in cricket is because it would be used to predict the path of the ball had someone's legs not interrupted it. Whereas in tennis it is simply used to account for where the ball actually went.
        Obviously just tracking a ball is a more definite science than the prediction of something that didn't happen (but could have). Especially as anyone who knows about cricket will tell you is that the path of the cricket ball is 'mysterious'.
        I once heard a cricket commentator interviewin
    • Can this be applied to something useful. You know besides whether or not someone was out in a game of tennis?

      Must of the technology that underlies Hawkeye was originally designed as a missile tracking system. Although personally I think sports are far more worthwhile than warfare.

  • major league base ball umpires union does not like systems like this and systems like that are not 100% also there stuff that is hard to make calls that can be 100% done by a bot.

    http://query.nytimes.com/gst/fullpage.html?res=9406E6DE1F39F933A15754C0A9649C8B63 [nytimes.com]

    http://query.nytimes.com/gst/fullpage.html?res=9D00E1D61130F933A1575AC0A9649C8B63 [nytimes.com]

    http://findarticles.com/p/articles/mi_m1208/is_24_227/ai_103378465 [findarticles.com]

    http://en.wikipedia.org/wiki/QuesTec [wikipedia.org]

    • The New York Times references are several years old. The Wikipedia article you mention says the controversy has died down and the system has brought the intended benefits.

      I wish they would just use an automated system in all the parks instead of relying on the umps. I also wish they would use a standard strike zone, instead of one that changes based on the batter. It'll never happen, though.

    • by KGIII (973947)
      Baseball would be FAR more difficult, if not impossible with a generic system, or at least that is my opinion. In tennis, football, etc. you have an exacting standard. Baseball fields are all different. Each one has different sizes, even the actual distance of the basepath may vary in distance though it isn't supposed to. Each player's height and stance is different meaning that there would be a great deal more difficulty if used to determine (in real time at least) the strike zone. I think the best solutio
      • by crossmr (957846)

        Review? are you kidding?
        Do you know how many "close" pitches there are in an average baseball game? I think they were working on lowering the average time of a game, this would shoot it way back up.

        • by KGIII (973947)
          You win. I hadn't thought of that. Though, again, I didn't mention that the review system for football should be used for baseball so I have a way to deny all accusations. *grins*

          (Really, I hadn't thought of that. Wow. That'd ruin it. I figured that that system had worked well for that game.) I played in college and actually wanted to consider an attempt at a career before admitting to myself how unrealistic it was and so I have a love affair with baseball so I'm at a loss of how we can automated the ref
          • by crossmr (957846)

            The only way to do it would be to develop the system so that is as accurate as possible. Sensors in the uniforms to measure the strike zone, sensors in the bat to determine if they check the swing, etc.

            If you can't get it to the same accuracy or better than an umpire, forget it.
            After doing that, you'd have to disallow reviews. That would speed up the game. The umpire NEVER changes his call when anyone argues with him, so people could storm out and have just as futile a conversation with the umpirebot 2000

    • by DriedClexler (814907) on Sunday June 29, 2008 @02:05AM (#23987935)

      I'm confused. Why would umpires oppose a technology that can automate the refereeing of a game? It just doesn't make any sense.

  • Anonymous Coward (Score:3, Insightful)

    by Anonymous Coward on Saturday June 28, 2008 @11:59PM (#23987393)

    They're reproducing stuff that's already known. Yes, Hawkeye can be inaccurate. However, it's MORE accurate than linesmen and certainly the chair umpire. That's why it's used as the definitive word.

    I'd certainly prefer it to be used otherwise - the best way would be to give the chair umpire the information from HawkEye and then let him decide whether to use it or not at any given time, properly educated about the types of errors the machine can make - but that wouldn't be as flashy, would it. So the advertisers wouldn't go for it.

    • by Lars T. (470328)
      The problem here is actually something else - which you can see by looking where the paper is coming from. Yup, Social Sciences are worried that people might think that any technological system could be perfect.
  • by Anonymous Coward on Sunday June 29, 2008 @12:04AM (#23987417)

    Hawkeye and the like deliver a consistent result. It matters not at all if the ball is in by two Centimetres but is called out, provided that error is consistent throughtout the match.
    If both players, or teams, are playing by the same margin of error, the contest is fair.
    In cricket for instance, I would accept the computers call over umpires any day of the week!

    • by Telvin_3d (855514) on Sunday June 29, 2008 @01:53AM (#23987885)

      That is only true if you assume the two players are making the same level of mistakes. If both players are regularly hitting the same shots witht he same amount of error, yes everything will even out. But let's say player A can consistently serve and hit the ball to within 2 cm of the out line, but player B often misjudges and goes 1 cm over. In this case, having player A's shots consistently called 'out' or player B's shots consistently called 'in' would be consistent, but it would also make a major change in the outcome of the match. And not the type of change that gets statistically evened out over games played.

    • by MtViewGuy (197597)

      The problem with line judges is that unlike Hawkeye, line judges have to see the landing point of tennis ball from pretty far away, and that tends to cause far worse judgment errors than with Hawkeye.

      No system is 100% foolproof, but those with the least amount of errors will reduce problems with judgment calls by by a long way. In fact, that's why Major League Baseball might even implement a replay system to determine home run calls as early as August 2008, given the recent spate of inaccurate calls by umpi

  • Ummm.... (Score:3, Informative)

    by Khyber (864651) <techkitsune@gmail.com> on Sunday June 29, 2008 @12:12AM (#23987451) Homepage Journal

    I've seen in Hockey and Football broadcasts the ability to track the ball or puck realtime thru some system inside the playing piece (puck or football.) It seems to work pretty decent to me.

    • It's not fully accepted in football ultimately because of the issues of ensuring the system could withstand the impact of being kicked. Bear in mind that these tiny tennis balls reach speeds >100mph; a system has to be able to withstand that impact.

      With hockey, I imagine that the puck is solid makes it simpler to ensure the integrity of the weighting of the puck after impact than it would be with a hollow tennis ball. Striking the relatively light and soft tennis ball to high speeds deforms the ball si
    • by Spasemunki (63473)

      It's certainly not used in NHL ice hockey. There used to be a "feature" in one of the network broadcasts of hockey games where they would add a "glow" around the puck to make it easier to follow on screen- this was done not using a tracker inside the puck, but was painted on digitally during the broadcast delay. Were a tracker to be put in the puck, the most significant use of it would be in deciding close goals where it isn't clear if the puck is over the line completely or not. No such technology is us

    • by barzok (26681)

      FoxTrax sucked ass. It often lagged the puck by a second or so and did not significantly help find the puck. Plus, it just looked cheap and lame. It was abandoned after a couple seasons.

  • by Zackbass (457384) on Sunday June 29, 2008 @01:05AM (#23987691)

    For those that didn't care to RTFA, the study is in the journal 'Public Understanding of Science' and (gooly who would have guessed) doesn't have anything to do with the summary written. They argue that uncertainties in measurement that normally don't impact the layman now need to be presented in an understandable way. They worry that people will wrongfully become too trusting of the systems that do have appreciable error in rare circumstances.

    To inject my own opinion on the matter, I've had a little bit of experience with Vicon motion capture systems which appear to use similar technology to the Hawkeye system. The main problem with the system (when it works) isn't any problem with accuracy or precision. In fact, it's awesome. The problem is that the output is a little noisy and suffers from occasional jumps and hiccups. With proper filtering these are eliminated and the output is amazing. I can only imagine the problem is much easier when you're tracking a single ball rather than tens of tiny reflective makers such as with the Vicon system.

  • It doesn't matter how good, or even how bad the system is. As long as it cannot be shown to have a particular bias to a player, or the side of the court then it is automatically more fare than any existing judge.

    period.

    It doesn't matter if it is even out my 5 centimeters, never mind having an error rate of less than half a centimeter.

    • by mdmkolbe (944892)

      An overly generous or overly tight judge will inherently favor either risk taking or conservative players. Thus there will always be a bias.

      Now the same could be said of umpires, but with umpires everyone understands the sorts of errors they introduce. The idea that the umpire could be wrong is freely discussed and lived with. As the article points out, systems like HawkEye usually foster the public perception that they are the absolute truth when in fact every physical measurement has a statistical e

  • by SamP2 (1097897) on Sunday June 29, 2008 @02:17AM (#23987993)

    For a system like Hawkeye to be useful, it doesn't need to be perfect. It just needs to consistently be more accurate and impartial than a referee can be.

    Nor is it required for the system to be fully automatic and autonomic. Referees can sit in front of their monitors, observe the cameras from all angles, with any time slowdown, and ultimately come to a better decision than a single person could make while the ball buzzes past them at Warp 9.

    But from the social aspect, one has to decide on what is the referee's role, and what kind of influence, if any, do we want to delegate to a computer. And that depends on the type of sport.

    For non-interactive sports such as sprinting, an automatic system works very efficiently, and most people readily accept it as better than a human time tracker.

    But for many GAME sports (soccer and boxing come to mind) many people consider that a referee is PART of the game rather than just an observer. As long as a referee is comparatively competent, and acts in good faith, he has the authority to judge events in the game, and while mistakes are unavoidable, they are considered part of the game as well.

    I'm not sure why this position is popular in these kinds of sports. Maybe it's the whole "humans should be judged by humans and not machines" aspect. Or maybe it's because having a Review Comission in front of CCTV monitors be judging every little move just feels too 1984-rish for spectators and players alike. Or maybe its something else. But this is a rather popular feeling.

    Depending on the features and benchmarks of the electronic system, it may or may not be more accurate than a human observer. In the long term, a joint human-computer analysis system would be certainly more accurate than a human referee alone, especially in team or high-speed sports. But one has to ultimately question, whether, by gaining mathematical precision, we lost some human touch of sport that makes it enjoyable to play and watch. Fun can't be generated with a mathematical formula. And sometimes sitting on the couch and thinking "OMG that referee is such a dumbass" is part of the fun as well.

    • Re: (Score:3, Informative)

      by jez9999 (618189)

      But one has to ultimately question, whether, by gaining mathematical precision, we lost some human touch of sport that makes it enjoyable to play and watch. Fun can't be generated with a mathematical formula. And sometimes sitting on the couch and thinking "OMG that referee is such a dumbass" is part of the fun as well.

      Watched the Aussie Open or Wimbledon in the last couple of years? I, and most other observers, consider that Hawkeye makes the game more enjoyable, and whilst probably isn't 100% accurate, i

    • by kmsigel (306018) *

      >For non-interactive sports such as sprinting, an automatic system works very efficiently

      The system (photo finish) used in sprinting isn't exactly automatic. A line scan image is taken of the finish line as the runners cross and a human looks at the image to determine in which frame the chest of a runner first appears.

  • All this strikes me as a more-or-less semantic argument. Yes the system has flaws, and yes it's the best we've got. Nothing to see here...
  • I am not sure how many cameras are used in Tennis, but I am guessing they use at least 6 of them to triangulate which I believe produces an accuracy better than that a human can produce.

    IIRC, Hawkeye was introduced in cricket to judge LBWs (Think of it as the ball hitting the batsman on it's way to the stumps). Now the rules say that any doubt must go in favour of the batsman, but obviously common sense prevails and umpires make a subjective decision on whether or not the ball is going on to hit the stumps.

  • I am not sure why people put technology to higher standards than humans. TFA (yes! I read it!) says this:

    Led by Professor Harry Collins and Dr Robert Evans, the team argues that such devices could cause viewers to overestimate the ability of any technological devices to resolve disagreement among humans. It also suggests that a more detailed understanding of how the device works could play a vital role in public education the benefits of which could spread to all technological decision making in the public domain.

    I am not sure Hawkeye CAUSES people to think machines are infallible, but it's rather backwards where HAwkeye is assumed to be infallible BECAUSE people expect computers and machines to be correct 100% of the time. The potential for error by a computer is alien to most people (not least because people like my idiot CS teacher in school keep telling you that computers are 100% accurate and that they never

  • by csoto (220540) on Sunday June 29, 2008 @10:03AM (#23989965)

    That ball was on the line!

To the systems programmer, users and applications serve only to provide a test load.

Working...