Slashdot is powered by your submissions, so send in your scoop

 



Forgot your password?
typodupeerror
×
Transportation Software

Researchers Trick Tesla Into Accelerating 50 Miles Per Hour (technologyreview.com) 171

mrwireless writes: Security researchers were able to get multiple Tesla cars to accelerate to 85 miles per hour on a road with a speed limit of 35. They did this by simply modifying the speed limit sign with some black tape, turning a "3" into an "8." Mobileye, the company that produces the Mobileye Eye Q3 for many of Tesla's cars, downplayed the research by suggesting that the modified sign would fool even a human into reading 85 instead of 35. "Autonomous vehicle technology will not rely on sensing alone, but will also be supported by various other technologies and data, such as crowdsourced mapping, to ensure the reliability of the information received from the camera sensors and offer more robust redundancies and safety," the Mobileye spokesperson said in a statement.

MIT Technology Review: "Tesla has since moved to proprietary cameras on newer models, and Mobileye EyeQ3 has released several new versions of its cameras that in preliminary testing were not susceptible to this exact attack."
This discussion has been archived. No new comments can be posted.

Researchers Trick Tesla Into Accelerating 50 Miles Per Hour

Comments Filter:
  • Common Sense (Score:5, Insightful)

    by Anonymous Coward on Wednesday February 19, 2020 @08:27PM (#59745414)
    People have common sense and a sign saying 85 in a country where nowhere has such a limit would fail the smell test and be rejected. A piece of hardware that interprets everything literally would only reject such a sign if it had explicit contradictory info. So no, comparing it to able to visually fool a person is a strawman.
    • People have common sense...

      "Nearly 1.25 million people die in road crashes each year"
      https://www.asirt.org/safe-tra... [asirt.org]

      I've noticed that my phone seems to know the speed limit better than I do.

    • That's okay, the driver will take a look at the road when the orange falls from the steering wheel and notice in time. Just like with firetrucks.

    • by ceoyoyo ( 59147 )

      Since the Tesla didn't go 85, perhaps it was doing just that. I wouldn't be surprised if there were people who would go 50 in most 35 zones, knowing full well what the actual limit was.

  • I doubt too many people will be fooled into believing an area with a 35 limit has an 85 limit (especially when nowhere int he country has that limit).
  • 50 miles per hour is not a unit of acceleration. One cannot "accelerate 50 miles per hour" like the OP's subject suggests.

    • by lsllll ( 830002 )
      Yeah, the quality of the summaries is going down, but you get the gist of it.
    • by chrism238 ( 657741 ) on Wednesday February 19, 2020 @10:58PM (#59745776)
      Accelerate [by] 50..... It's not difficult to accept a small typo.
      • by Pascoea ( 968200 )

        No, it really isn't that hard to surmise what the editor was getting it. But, it gets a little harder to accept when: a) Their literal job title is "editor" b) It happens a lot.

        Everybody has sent (likely multiple) e-mails with typos, improper grammar, etc. etc. in them. But how many times is the boss going to accept them in company-wide presentations? I'd think after the 2nd or 3rd you'd get a pretty good talking to.

        • Re: (Score:3, Insightful)

          by rtb61 ( 674572 )

          It is a criminal offense to deface a traffic sign, even temporarily, I hope they had of the permission of the local constabulary, if they did not, they should be prosecuted, as a warning to others.

          • by fgouget ( 925644 )

            It is a criminal offense to deface a traffic sign, even temporarily, I hope they had of the permission of the local constabulary,

            Based on the screenshot of the video in the fine article this was done on privately owned speed sign in a private parking lot.

      • without reading the summary the headline makes zero sense
  • Not True (Score:5, Insightful)

    by crow ( 16139 ) on Wednesday February 19, 2020 @08:46PM (#59745480) Homepage Journal

    The Tesla will never accelerate above the speed at which the Autopilot was initially set. For example, you could set the cruise control at 55, then engage Autopilot, which will limit the speed to 5 over the posted limit unless it's a divided highway. So if the speed limit drops to 35, the car will slow to 40. If the speed limit then jumps to 85, the car will accelerate to 55. If for any reason you canceled the Autopilot and took over manual control and resumed it, then the active limit would be down to 40, so it would not accelerate at all when seeing the sign saying 85.

    Autopilot ignores the speed limits if it's a divided highway, so this only matters for local roads. On local roads, drivers will be canceling and restarting Autopilot at traffic lights or for many other reasons. Reading speed limit signs incorrectly is a little annoying, but hardly a serious problem. The worst is when it sees a lower limit sign that doesn't apply (like a school zone out of hours or a frontage road with a lower limit), and the car slows down when it shouldn't. But these are all local roads where drivers have to be paying attention, as there are far too many things the system doesn't handle anyway.

    Oh, and I've had it misread a sign without any tape and briefly display on the dash that the speed limit was 85mph. It was funny, but the car didn't speed up.

    It's no big deal. Unless you're looking for an excuse to drive a lot of traffic to your web site.

    • and when tesla auto pays tickets cops will fix it to rake in the cash.

    • Reading speed limit signs incorrectly is a little annoying, but hardly a serious problem.

      Sure, why even have speed limit signs at all?

      • There's actually a good argument to be made for eliminating speed limit signs. I've driven in places where traffic enforcement is nonexistent; didn't see any real issues. It definitely wasn't as safe as driving in a first-world nation, but the extra danger didn't come from people driving at unsafe speeds so much as it did from them ignoring stop signs and traffic lights. People tend to adapt to road conditions, and traffic often makes the posted limit irrelevant anyway.

        If we ever do get to the point tha

        • by Cederic ( 9623 )

          There's actually a good argument to be made for eliminating speed limit signs. I've driven in places where traffic enforcement is nonexistent; didn't see any real issues.

          My last speeding fine was in Morocco. Left a small village, accelerated up towards the national speed limit, turned a corner, missed the 60km/h limit sign, went past a school at 80km/h.

          I didn't know there was a school just outside the village. The schoolkids didn't know a random tourist would be driving through their obscure part of Morocco and driving on the assumption it was open desert.

          That speed limit sign added value. About twelve quid of value to the police, as well as hopefully reducing traffic casua

          • by kenh ( 9056 )

            My last speeding fine was in Morocco. Left a small village, accelerated up towards the national speed limit, turned a corner, missed the 60km/h limit sign, went past a school at 80km/h.

            So in Morocco they don't post school zone signs, they just tuck their schools back off the roadway and rely on speed limit signs to protect the children? The posted speed in a school zone was 60 Km/h, or 37 MPH? That seems kinda fast - here in America we tend to limit school zones to 25 MPH and post school zone signs.

            • by Cederic ( 9623 )

              Maybe I was doing 60 and should've been going slower? It was a while back.

              They don't post speed change signs, which is why I thought I was outside of the village reduced speed zone.

              Rather than school zone signs they paint the schools in bright stripes. Makes them obvious and visible from quite a distance. Unless you come around a corner up a hill.

    • by McWilde ( 643703 )
      I don't know about Autopilot, but cruise control in the model 3 will certainly accelerate to speeds above where it was set if the car thinks the speed limit has changed. Example from the one time I was a passenger in a Tesla: driving through a construction zone where 90 km/h was allowed and set for the cruise control the car suddenly decided the limit was 40, slammed on the brakes, then decides it's actually 130 and starts accelerating hard until it notices the car in front and hits the brakes again. Not be
      • by crow ( 16139 )

        Tesla Cruise Control ignores speed limits. Unless you activate Auto Steer (i.e., Autopilot), it will do whatever speed you set it at, and it won't change when the limit changes.

    • by AmiMoJo ( 196126 )

      The way this attack would work is the victim sets the cruise speed to say 85, then the car comes to a slower section of road where the limit is say 55. Normally the car would see the sign and slow down, but today someone put tape on the sign and it decides that it will carry on at 85.

      Of course the other safety systems will be in play like front facing radar, but those can't protect against things like trailers and some types of truck. Or speeding tickets for that matter.

      In theory the driver should be paying

      • by kenh ( 9056 )

        This isn't autopilot, this is cruise control.

        • by crow ( 16139 )

          No if you just use cruise control, it doesn't adjust for speed limits at all. The Auto Pilot will, but will never increase above the last manually set speed.

      • by crow ( 16139 )

        The situations where you would set the cruise control to 85, engage Auto Pilot, then enter a 30mph zone, slow down to 35 (it goes 5mph over), and then enter a 35mph zone with an altered sign and accelerate to 85 as described in the article are all quite contrived. Yes, it's theoretically possible, but no, this isn't the sort of scenario that is going to impact real drivers.

        • by AmiMoJo ( 196126 )

          Wouldn't have to be a 30 zone, even a 60 zone would give you a delta of 25 MPH and attract the attention of speed cameras and cops.

          Also the 5 MPH over thing is not a very good idea in the UK. You can be convicted of speeding by doing 35 in a 30 zone.

    • by GoRK ( 10018 )

      Also, Tesla doesn't read speed limit signs on anything except Mobileye-based AP1 due apparently to Mobileye patents. Reading speed limit signs is basically a machine learning party trick these days, so it's not like they can't; they just don't.

  • by TheGratefulNet ( 143330 ) on Wednesday February 19, 2020 @08:50PM (#59745486)

    tesla stopped using mobileeye a LONG time ago.

    so, this article is FUD. its a statement on M.E. and not tesla.

    that said, M.E. is still better software, but as for this article, its just more pot shots at tesla. it seems fashionable to try to mention tesla in any way you can..

    • by DRJlaw ( 946416 ) on Wednesday February 19, 2020 @09:37PM (#59745590)

      tesla stopped using mobileeye a LONG time ago.

      All those pre-2017 Tesla's sold with MobileEye technology? Obsolete. Take them to the electronics recycler.

      so, this article is FUD. its a statement on M.E. and not tesla.

      It's a statement on Tesla. Tesla selected MobilEye for its HW1 technology. Tesla pushes updates to vehicles with HW1 technology.

      that said, M.E. is still better software, but as for this article, its just more pot shots at tesla. it seems fashionable to try to mention tesla in any way you can..

      Name any other vehicle deployed to consumers that attempts to read speed limits off of road signs. It's not a pot shot. It's a valid criticism of a design flaw.

      • Every other vehicle in the world that isn't driven by auto pilot attempts to read speed limits of road signs. The outrage here is absurd. I'm actually wondering if this wasn't sponsored by the onion.

      • "Name any other vehicle deployed to consumers that attempts to read speed limits off of road signs."

        Pretty much any modern car - BMW, Audi, Mercedes... they have cameras, sign recognition, adaptive cruise control.

  • This OP is lying (Score:5, Informative)

    by Strill ( 6019874 ) on Wednesday February 19, 2020 @08:53PM (#59745492)

    They didn't turn a '3' into an '8'. They just lengthened the middle prong on the three. It's still obviously a three. Take a look yourself.
    https://cdn.technologyreview.c... [technologyreview.com]

    These Mobileeye people who are saying that real people would be tricked by this are a bunch of liars.

    • by Xenx ( 2211586 )
      I agree they're trying to downplay it by saying people would be tricked, but I do believe it would trick some people visually. However, I believe that few of those people that read it as 85 would actually believe the speed limit is 85.
    • That's a nice closeup of the speed limit sign. And yes people would easily interpret that as an 8 at a distance or at a glance. It sure as heck doesn't look like a 3.

      Incidentally that's relevant. No sane person drives 85mph on a 35mph road, they are afterall 35mph for a reason.

  • by whoever57 ( 658626 ) on Wednesday February 19, 2020 @09:16PM (#59745542) Journal

    The full truth about this is that the cars they tricked were older model cars that were built with a chip that is no longer used by Tesla.

    More recent Tesla models don't attempt to read speed limit signs. Instead, they rely on mapping data.

    The mapping data has it's own issues: it has been suggested that one cause of the phantom braking issue is streets running over or under the freeway that have lower speed limits.

    • Re: (Score:3, Insightful)

      by DRJlaw ( 946416 )

      The full truth about this is that the cars they tricked were older model cars that were built with a chip that is no longer used by Tesla.

      More recent Tesla models don't attempt to read speed limit signs. Instead, they rely on mapping data.

      The full truth about this is those "older model cars" are well within their service lives. Ford didn't get a pass on the Pinto simply because they were older model cars replaced by the Fiesta.

      More recent Tesla models don't attempt to read speed limit signs. Instead, they

    • Do those older model cars suddenly disappear from the roads? Hand waving them away doesn't remove the potential problem for cars on the road today or even tomorrow.
      • Hand waving them away doesn't remove the potential problem for cars on the road today or even tomorrow.

        No, but an over-the-air firmware update may fix the issue.

  • by AnotherBrian ( 319405 ) on Wednesday February 19, 2020 @09:44PM (#59745604)
    On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. —Charles Babbage, Passages from the Life of a Philosopher
    • by kenh ( 9056 )

      Researchers discover "Garbage In, Garbage Out" - news at 11:00

    • On two occasions I have been asked, "Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?" ... I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. —Charles Babbage, Passages from the Life of a Philosopher

      Awesome quote, I love it. I'm going to use it frequently. I may make a t-shirt with it, though it's a little long for a t-shirt.

  • Speed limit DB + GPS has issues as well
    Like some times an GPS can say you on frontage road when you are really on the main highway or even on road that is over passing you.
    Tunnels / Mounting areas.

    Now an Speed limit DB can get stale
    May not have the detailed level needed. Like at mile 1.3 WB is 55 and EB is 65. Or the road changes speed half way from the next intersect point.
    Also how many paid mods will you need just to keep up with level of data in 1 metro area?? Out source it to place where workers have no

    • I have speed limits displayed by both my Honda Insight's dashboard and on my Garmin Drive GPS. They sometimes mismatch... the car display is easily confused by things like a State Route 70 sign just outside the dealership causing it to display Limit 70. The GPS screen is more accurate... just please don't link these things to automated enforcement systems.

  • by CohibaVancouver ( 864662 ) on Wednesday February 19, 2020 @10:04PM (#59745658)

    able to get multiple Tesla cars to accelerate to 85 miles per hour

    *85* MPH?

    Everyone knows the flux capacitor won't engage until the car is up to 88 miles-per-hour!

  • "Researchers Trick Tesla Into Accelerating 50 Miles Per Hour"--- Wouldn't that be MPH per some unit of time? So much for a "sci/tech" site...Before someone throws the pedantic card on this, I was wondering how long it took to do that. I know, read the friendly summary...
  • My god, what will it do when it comes across one of those "ONE LANE" signs that the hippies have changed to say "ONE PLANET"?

  • In fact, my car sometimes accelerates to 85mph even when no one has messed with the sign.
  • Please, for the love of god Slashdot, quit posting bullshit articles.

  • Now replace it with an 85 km/h sign and see how humans do!

  • by nospam007 ( 722110 ) * on Thursday February 20, 2020 @01:52AM (#59745994)

    ...2351 old grannies, 3543 gramps, 17632 people without their prescribed glasses on and the rest ignored speed limits on general principle.

  • The image in the article CLEARLY shows that the middle section of the 3 was extended, making it still recognizable as a 3. I expect a written apology.

  • kidding much (Score:5, Interesting)

    by Tom ( 822 ) on Thursday February 20, 2020 @05:45AM (#59746220) Homepage Journal

    They did this by simply modifying the speed limit sign with some black tape, turning a "3" into an "8."

    You're kidding, right?

    So they showed the car camera a sign that says "85" and then act all surprised when it believes that there was a sign that said "85" ?

    Yes, I saw the actual picture. Most humans would not misread it. The software does because its recognition works differently from ours.
    Here's the thing: We all know that image recognition is very successful, but has edge cases. The school bus mistaken for a giraffe, etc.

    It's an interesting hack, but it's basically just saying "look, we found way 1001 how to trick image recognition software". Yes you did. Here's your badge, now take a number and stand in line.

    The only reason this is a news item at all is because "Tesla". Seems not all the shorters were squeezed out yet and we're back to "look at what things Tesla gets wrong (and let's not talk about other cars)". I mean, I drive a BMW and love it, but is traffic sign recognition sometimes fails on signs that weren't even manipulated in any way. I don't see news articles about that, and my best guess is that it's because nobody with deep pockets is shorting BMW.

    • They did this by simply modifying the speed limit sign with some black tape, turning a "3" into an "8."

      It's an interesting hack, but it's basically just saying "look, we found way 1001 how to trick image recognition software". Yes you did. Here's your badge, now take a number and stand in line.

      As someone who works with safety critical engineering systems, this raises concerns to me far beyond the image recognition software. It tells me that the Tesla Autopilot software is relying on a single source of data to set a safety critical parameter (speed) - a huge no-no in any safety critical system (see 737 MAX). Even if the sign were read as 85mph, a good driver would evaluate his surroundings and the road conditions before deciding whether to accelerate. Common sense would dictate that a sign indicating a speed well above highway speed levels in a small rural road should not be taken at face value.

      One should not be using a single source of data as a key input - and whether that data comes from image recognition software or the mapping software, the basic problem remains that the Tesla Autopilot is vulnerable to a critical safety issue if a single data input is corrupted. Ideally they should be having a combination of image recognition and mapping data, coupled with safety monitoring software to invalidate transient spikes in the data and obviously erroneous values.

      • by Tom ( 822 )

        Even if the sign were read as 85mph, a good driver would evaluate his surroundings and the road conditions before deciding whether to accelerate. Common sense would dictate that a sign indicating a speed well above highway speed levels in a small rural road should not be taken at face value.

        Agreed to that.

        Now if we had C-ITS in place already, the car would have a digital signal from the sign as well, and speed information from the map data, and could play the usual "two votes beat one vote" game.

        Though the car does take traffic around it into account, so you wan't be able to make it do crazy things during rush hour.

        coupled with safety monitoring software to invalidate transient spikes in the data and obviously erroneous values.

        Roads are messy things. My car does the same thing except for the auto-driving, it only uses the signs to inform me what it thinks the speed limit is.

        Half of the time it's wrong. It

    • by kenh ( 9056 )

      Next up, researchers are able to "trick" street lights into staying on 24 hrs/day by taping over light sensor.

      After that, we'll learn that researchers were able to "trick" a grocery store doorway into staying open by placing a gym weight on the mat.

  • Tape the license plate, too. Traffic monitoring cameras will not be able to identify your car.

  • by kenh ( 9056 )

    First off, pretty sure vandalizing a traffic sign is a crime, so this is illegal.

    Second, am I to understand that Tesla's will accelerate to the posted speed limit in all circumstances? That seems like the real story.

    Third, so what? I mean seriously, do we honestly expect your car to keep an updated map of the entire country (or better, planet) with an accurate database of the maximum speed limit on every street? Because that is the only alternative to the car relying on highway signs and other indicators.

  • by ssyladin ( 458003 ) on Thursday February 20, 2020 @12:07PM (#59747146)

    " ... most people aren't murders"

    https://www.xkcd.com/1958/ [xkcd.com]

  • by zenintrude ( 462825 ) on Thursday February 20, 2020 @12:24PM (#59747202)

    He managed to trick a road runner by painting a tunnel on the side of a mountain

    https://youtu.be/4iWvedIhWjM [youtu.be]

Top Ten Things Overheard At The ANSI C Draft Committee Meetings: (10) Sorry, but that's too useful.

Working...