NHTSA Tells Tesla To Stop Exaggerating Model S Safety Rating 284
cartechboy writes "There's always that kid in the class that ruins it for everyone when being graded on a curve. At the moment, that kid is Tesla and Elon Musk. Tesla's been proudly claiming the Model S is one of the safest cars in the word despite the recent fire controversy. And while it may be just that, claiming it earned 5.4 stars from NHTSA isn't pleasing the safety agency as there is no such thing as a rating higher than five. While NHTSA already released a statement indirectly to Tesla saying it doesn't release ratings higher than 5, Tesla continued to promote this fictitious rating. Now NHTSA has updated its guidelines explicitly stating safety ratings are whole numbers only and that 5 stars is the maximum advertisers can claim. If advertisers and automakers decide to disregard these rules NHTSA is threatening removal from the program or referral to state authorities for appropriate action. Basically, hey Tesla, stop making false claims."
Genius and insanity go hand in hand (Score:5, Interesting)
So how did he get a 5.4? (Score:3, Interesting)
Anyone have any explanation on how he got a 5.4? I don't think Musk would just arbitrarily post a number.
Re:Misleading (Score:4, Interesting)
Re:Misleading (Score:5, Interesting)
There is no analytical basis to explain exactly how 5.1 is less safe than 5.4 and the analysis makes no such claim. If the NHTSA allowed manufacturers to abuse the figures by claiming these fractions are meaningful then the rating system would lose credibility.
If that's true, then there's also no analytical basis to explain how exactly 4.4 (Which you rounded down to 4) is less safe than 4.6 (Which you rounded up to 5).
The fact of the matter is, if such is your degree of error, then You have to make distinctions for which there is no analytical basis;At some point, if you are truncating the number: you have to choose which whole number you will truncate it too, AND another figure that is only 0.1 different, will appear as a whole POINT less safe or more safe than the other --- while other numbers that differ by 0.4 or more, will be truncated to the same value.
Listing the more detailed figure is not per-se an abuse, then.
There is false precision there; and there is also false precision truncating the number+fractional part to one of the surrounding whole numbers.
Conveying the decimal point conveys false precision, BUT at the advantage of eliminating bias and haphazard truncation.
You can actually see that one car got 4.4, and another got 4.6; instead of seeing "4 and 5"; Off by a whole star ------ there is something to hand you a relative showing, that they're really rather close, maybe the same.
Re:Misleading (Score:3, Interesting)
There is false precision there; and there is also false precision truncating the number+fractional part to one of the surrounding whole numbers.
Conveying the decimal point conveys false precision, BUT at the advantage of eliminating bias and haphazard truncation.
You can actually see that one car got 4.4, and another got 4.6; instead of seeing "4 and 5"; Off by a whole star ------ there is something to hand you a relative showing, that they're really rather close, maybe the same.
The problem isn't false precision. It's not even false accuracy (which is what I think you meant). It is the conceptual basis on which the safety scores are based. The NHTSA specifies certain criteria and example accident scenarios and then computes a score. This score only strictly speaking applies to the particular accident types covered by the testing. Clearly the testing is going to be reasonably representative, but it will not be possible to design a test suite that is fully representative of all possible scenarios encountered in the real world.
The NHTSA doesn't want manufacturers to optimise for the particular suite of sample accident scenarios to gain an extra 0.1 score and beat their rivals, because that would not mean that the real-world safety was improved and might even mean that safety declined slightly in non-tested accident scenarios. By rounding the scores it eliminates the motivation for this pointless effort, for all but a few borderline cars, and encourages manufacturers to focus only on possible major safety features that have a meaningful impact.
Tesla are also facing criticism for doing a similar thing in accounting - designing their own 'non-GAAP' measures of financial performance and highlighting them, with the 'GAAP' measures less prominent. Again, it's not that their 'non-GAAP' measures are not accurately computed or based on false data, its that they measure things that aren't necessarily meaningful to investors.
Re:He'll love that (Score:5, Interesting)
It reminds me of when the laboratories that rated sunscreens could only rate them up to SPF 15, and pretty much every sunblock on the market was exceeding that. At some point, at least in New Zealand and Australia, the government stepped in and said they couldn't advertise higher than 15, so they all became SPF15+ for a while.