Do Car Safety Problems Come From Outer Space? 437
Hugh Pickens writes "As electronic devices are made to perform more and more functions on smaller circuit chips, the systems become more sensitive and vulnerable to corruption from single event upsets. This is especially true of Toyota, which has led the auto industry in its widespread inclusion of electronic controls in the manufacture of their various car models. 'These circuit families store not just data, but their basic function electrically,' says Lloyd W. Massengill, director of engineering at the Vanderbilt Institute for Space and Defense Electronics at Vanderbilt University. 'In the unfortunate event of a particle flipping just the right bit, a circuit configured to carry out a benign action may be reprogrammed to carry out some unintended action.' Denise Chow writes in Live Science that some scientists are pointing to cosmic ray radiation as a plausible mechanism behind the sudden, unexplained acceleration reported to have occurred with the late model Toyotas."
"As the design of automobile systems continues to evolve from mechanical to electronic controls, relying more and more on various circuitry and chips, these electronic components may be vulnerable to being confounded by high-energy radiation writes Chow. Federal regulators were prompted to look into the possible role that cosmic rays played in Toyota's product recall fiasco after an anonymous tipster suggested the design of Toyota's microprocessors, software and memory chips could make them more vulnerable (PDF) to interference from radiation compared with other automakers. 'What's not known is what direction Toyota and other automakers are taking in terms of finding and correcting these issues,' says senior researcher Ewart Blackmore."
Why they tell you to turn off your phone... (Score:5, Informative)
Interference from radiation doesn't just come from outer space, it comes from cell phones, TV/radio stations, microwaves.... you see where this is going. I once worked in an office where there was a cell phone relay antenna too close to a PC, and we were constantly reinstalling the OS until I told them to move things around in the area.
Thing is, when Windows gets a corrupted OS... it BSODs and we move on. Single-bit errors shouldn't send the car out of control... there should be some checksum that shouldn't add up. When a fault is detected, it should go to a backup program about safely shutting down the car.
Re:Why they tell you to turn off your phone... (Score:5, Informative)
http://en.wikipedia.org/wiki/Non-ionizing_radiation [wikipedia.org]
Granted, an unshielded circuit can be vulnerable to any EM field, but gamma rays affect electronics in a completely different way than microwaves do.
Re:Sun UltraSPARC-II's anyone? (Score:2, Informative)
I work with someone who used to do tech support for Sun - those flips were due to a manufacturing error - tech support were just told to tells customers it was due to 'Sun Spots'.....
Re:Is there realy a problem? (Score:1, Informative)
And the trial lawyers involved in the class action lawsuits being filed against Toyota have a vested interest in showing otherwise.
Re:Why they tell you to turn off your phone... (Score:3, Informative)
Nope, the exact opposite. Gamma rays [wikipedia.org] are short wavelength and high energy.
Re:Why they tell you to turn off your phone... (Score:4, Informative)
If red cars are an indication of the problem, it's more widespread than engineers used to believe. On a more serious note: Fault tolerant design is the answer. Have three systems calculate the result (ideally using three different algorithms) and let them vote on the correct result. Don't assume that a set state persists, recalculate frequently and set the state even if it should be already set. Feed the control and the sensor data into a watchdog circuit (in triplicate...) to detect mismatches. Etc.
Re:Why they tell you to turn off your phone... (Score:3, Informative)
Gamma rays have a higher frequency,
Corrected. And thus they have a shorter wavelength.
Re:Is there realy a problem? (Score:3, Informative)
Since the biggest Toyota runaway story has turned out to be a problem exists between seat and pedals situation...
The article you linked to does not even begin to support that conclusion. Basically its a bunch of innuendo, like he [i]might[/i] have been late on payments on the car (since proven false) or that he should have shifted it to neutral (not an intuitive action for someone who has never driven a manual transmission - and certainly a last resort that does not negate the existence of a problem to begin with). Even information released after that article was published has been far from damning - basically toyota has said "we couldn't reproduce the problem" - as if "works for me" means there are no software bugs.
The undisputed facts are that the brakes were severely worn (although Toyota claims that the wear is not consistent with emergency braking - huh?) and that the car's black-box showed that the guy hit the brakes over 200 times during the time of the incident and that a cop witnessed the guy practically standing on the brakes.
Unless there is more that's come out recently, all facts released so far point to a failure with the car, not the nut behind the wheel.
Re:Why they tell you to turn off your phone... (Score:5, Informative)
The proposal by the GP poster is actually much more difficult that it would seem at first glance. About the only place "checksum" style error detection is used is in memories/registers. The reason is that if I do a floating point addition, for example, the only way I know whether the addition gave me the right answer is to do the addition again and check.
Re:Is there realy a problem? (Score:1, Informative)
Re:No. (Score:5, Informative)
Actually, every CPU _IS_ vulnerable to bit-flips from radiation. That part of it is not speculation. It does occur in commodity processors, and with probabilities large enough that we have ECC ram, and ECC and/or parity in caches. Some servers actually come with built in hardware fault tolerance methods, because when you run hundreds of servers non-stop for years, the probability that a particle strike screws up a register on chip is non-negligible. Now, still, the probability isn't _huge_. Definitely not high enough to be causing these specific problems, especially when the failure is always in the same manner. _That_ part of it is pretty much bullshit.
Re:Why they tell you to turn off your phone... (Score:4, Informative)
You can build circuits that detect faults while operating. They're more complex than their normal counterparts, but the transistor count is less than 2x. On-line error detection [google.com] is a common name.
Of course, such circuits get really expensive if you don't have a large market for them. But cars represent a fairly large market, so if it was the best approach they could probably use them. Of course, that assumes there's any market or regulatory pressure to use any sort of error detection at all.
Frontline Auto Engineer's Perspective (Score:5, Informative)
My little part of ETC involved adding a sub processor which watch-dogged the main micro. The little micro asked a series of questions of the main micro. Both processors would need to agree on all the inputs and output of the system. The little micro would also ask question regarding real time OS (RTOS) of the main micro. The main micro would need to have tasks executing in the right order to satisfy the small micro. Lastly, the small micro would ask the main micro to perform math operations to verify accuracy. Oh, and the main micro was continuously checksumming it's memory too.
Both micros had a direct hardware disable path to the H-bridge which was delivering power to the throttle plate. The throttle plate was spring loaded, so, with power cut, the throttle plate would snap to an idle position.
Next came the electro / magnetic compatibility testing (EMC). We spent months inside huge chambers testing both radiation and susceptibility. One of the tests for susceptibility involved using a zap gun to spark a 20kV spark on each pin of our ECU. Not satisfied with that, our customer opened one of our modules and used a sparking spark plug to slowly zap our board to failure. Bottom line, that throttle plate better never stick one way, or the other.
In the end, it always amazed me that the whole thing would work at all. Seemed to me that the system was always seconds away from going into some kind of fail safe mode.
No, a stray bit flip is not going to facilitate a run away car. Least not on my system!
Re:Why they tell you to turn off your phone... (Score:5, Informative)
I worked on ECMs at GM (Delco Electronics) for 10 years at the start of their use (1980 to 1990). So if a cosmic ray came along and flipped a bit, it would have to be a specific bit. If it was a msb type bit in the accelerator position, then yes, acceleration. except that the bit would unflip right away because of pedal position update. Or if it was some engine feedback msb, again, yes, temporary acceleration, but again, only for a short time. Updates happen constantly.
About EMI/EMC/RFI - the modules have been shielded and protected since day one against that. The engine is a very high disturbance environment in may ways. Sparks, for instance. The ECMs have been in almost all American cars since before 1980, because of the 1975 car air pollution reduction act Congress passed. The only way cars could meet the pollution restrictions was through ECMs. So If we have ECMs since nearly forever, and only just now one manufacturer has a bit flip problem? I don't think so. And these modules do not use the latest super-small feature processor technology. They use older temperature-resistant tech, Much larger features, far more radiation-resistant.
No, the most likely problem is either a software routine with a bug, no error handler, or similar issue, or a mechanical,problem (less likely).
Re:Why they tell you to turn off your phone... (Score:4, Informative)
Why post AC? You obviously work for NASA [wikipedia.org]. :)
Redundancy in a car isn't essential for the computer, as long as it fails in a safe mode. In the case of a single bit being flipped in the data stream, that would be a transient error. In a throttle system, it would be so short lived, you'd never know it ever happened. How many times per second do you think the computer reads its inputs and adjusts things? (hint: it's more than 1).
Heck, you don't even (usually) notice misfires, and those happen all the time, even on perfectly tuned vehicles. It takes a whole series of misfires, or a constant fault to be noticeable. On a V8 engine, you can even lose a cylinder and not notice. I had someone once bring a car to me because it "doesn't accelerate well". It turned out three spark plug wires weren't on. And no, I didn't work on it before that, someone else messed up. It actually idled pretty well. The three cylinders weren't sequential, so it managed fine. That's even been included as a feature on some cars. For example, an 8 cyl car would disable 2 or 4 cylinders to get better fuel economy, and run on all 8 if full power was requested. It's sometimes referred to as a variable displacement engine. Versions have shown up in GM, Chrysler, Mercedes, and Honda vehicles over the years.
Re:Why they tell you to turn off your phone... (Score:5, Informative)
The best chart of lead isotopes I found is here http://education.jlab.org/itselemental/iso082.html [jlab.org]. I'm not sure why, but it lists a half life for lead-204 even though I thought it was supposed to be stable. Most half lives are a few minutes or hours.
Re:Why they tell you to turn off your phone... (Score:3, Informative)
Boeing's 737 production since 1967: 6,285 aircraft
Toyota's production in 2007 alone: 8,880,000 vehicles
Re:Why they tell you to turn off your phone... (Score:3, Informative)
I don't hear much about comsumer electronics being fritzed by cosmic rays,
Chances are you'll be hearing about this more and more over the next several decades or so. Scientists have discovered a large spot over the Atlantic (IIRC) where high levels of cosmic radiation are actually making it to the ocean's surface. Further investigation indicates this is because their Earth's magnetosphere is beginning to significantly weaken. Furthermore, its expected that not only will the the level of radiation exposure continue to drastically rise at this particular location, but that radiation exposure globally will drastically rise.
It turns out, it appears this is related to the shifting of Earth's magnetic poles. As the poles continue to migrate away from their axial positions, the earth's magnetosphere begins to dramatically weaken. Not to surprising, the protection extended to both artificial satellites and Earth's occupants will be significantly and negatively affected.
Accordingly, expect far more electronics failures from cosmic radiation over the next several decades and beyond. And over the next thousand years, the levels of radiation may pose a significant risk to all life on Earth - or at least those on the surface. This of course, also suggests we will have a pole reversal sometime within the next thousand years.
Obviously far more research is required.