Taking a break from #Musk's Hate Train on the Hellsite to recall this series of Tweets from a few years ago.
While under-appreciated then and now, the Tweet thread by Musk posted below contains an extremely damning #SystemsSafety admission and it displays the considerable #PublicSafety blind spot associated with remotely updating #SafetyCritical systems without oversight.
Musk has no clue what he admitted to here, but systems safety experts do.
@adamjcook I bet the internal patch notes on those updates are wild.
“Fixed issue where pedestrians wearing green glasses were identified as traffic lights and vehicle continued driving into them”
How can we even think that treating software that controls 4000lb machines that interact closely with people should use the same development processes and quality concerns as a smartphone app? It is terrifying.
My thread is about how this series of Tweets from Musk reveal quite a bit about #Tesla's internal engineering processes and how troubling those processes undoubtedly are.
And if you hear anyone describing them as such, it almost certainly means that they are (knowingly or not) hand-waving away the incomparable #SystemsSafety differences between a consumer electronic device and a #SafetyCritical system.
The Hard Truth is that if one's engineering and management experience has been dominated by their time at, say, #Apple... this will not translate well to a role that involves #SafetyCritical systems of the caliber of cars.
The competencies involved are Night and Day.
That is not to say that talented engineers from consumer/business hardware and software realms can never become competent in safety-critical systems work... but it is a considerable jump.
First off, it should be recognized that a "regression" is a software term which, in the context of a #SafetyCritical system, is woefully incomplete by itself.
The question asked and the statement made should be...
"How was an existing validation process deficient such that it allowed a safety-related defect to enter the public?"
The second Tweet touches on a topic that Musk is undoubtedly unaware of called Configuration Management (CM) - an extremely complex and important concept in #SafetyCritical systems work.
You know sometimes, if you have an older #iPhone, and #Apple pushes an OTA update and in some older devices it causes some issues that newer devices do not see?
That is likely because Apple has lost some visibility on the significant number of hardware configurations that they are supporting.
But a potentially deadly situation when CM visibility is lost with a #SafetyCritical system like a car!
In that Tweet, Musk is hand-waving #Tesla's responsibilities in maintaining CM (and a validation process to match) as an "impossibility".
It is not impossible.
It is inherently costly and complex and it will substantially reduce Tesla's flexibility in changing vehicle hardware on-the-fly - an oft-cited competitive advantage.
Musk does not want that baggage, which is unavoidable in responsible#SafetyCritical systems work, so Musk and #Tesla toss "the testing" upon its untrained customers and the public.
That is why that Tweet is so revealing.
The other thing, of course, is that "QA", by itself, is not a sufficient processes for #SafetyCritical systems - and, yet again, it is a term stripped from consumer/business software and hardware domains.
The last Tweet in that thread was written by Musk a little over 13 hours after "the issues" were discovered when the second Tweet was published.
For #SafetyCritical systems of this complexity, no matter how many people are on the team, no matter how talented the people are on the team, there is zero chance that the "10.3.1" point update was actually validated.
There simply is not enough wall clock time.
Musk and #Tesla just tossed it out, like if they were shipping a video game update.
There is effectively no #automotive regulation in the US and, as such, there is no independent scrutiny on #OTA updates for cars.
OTA updates can yield public safety benefits in quickly and efficiently rectifying safety defects.
But the OTA update mechanism can also be used by automakers to water down upfront validation (in order to get to market faster and cheaper) and to hide safety defects without a #recall.
And, because there is no oversight on the latter, they should be disallowed.
@adamjcook They're pushing stuff live without validating safety, in order to deal with the consequences of pushing stuff live, without validating safety?
That's really, really bad.
Begging for some major event where things go very wrong.
@justafrog Naturally, not that you suggested it, I do not want to see anyone hurt or killed by #Tesla's wrongdoings - which are vast and uniquely extreme.
Rather, I would like to see regulators (who are supposed to represent the public) start to pay attention to these damning admissions as Musk makes them.
Regulators should be seeking to prevent avoidable injury and death and by ignoring these clear-cut signals from Musk... they are failing to do so.
@justafrog Your lack-of-faith for automotive regulators is completely valid.
No one should have any.
The #NHTSA, in the US, is a horrible regulatory agency - worse than can be possibly imagined.
The agency spends all day constructing elaborate, but paper-thin facades and puppet shows for the public - all while the public is being materially harmed.
And, indeed, they really only wake up, just a little bit, when a bad headline comes out that they cannot sweep under the rug.
@justafrog@adamjcook Calling it now: ransomware pushed to OTA converts all Tesla OSes to bitcoin miners, disconnecting the communication and ignition hardware, which causes the cars to overheat until they explode.
@adamjcook@samabuelsamid reminds me of my 2021 Mach-E, whose High Voltage Junction Box was prone to failure, leaving the vehicle bricked. Ford eventually recognized the problem, but instead of replacing all affected HVJBs, they did an OTA "recall" which did not in ANY way fix the problem, just allowed the car to run in turtle mode so u could drive it to a dealer. But most owners think the OTA fixed the problem, and Ford escaped scrutiny for not actually fixing the problem.
@adamjcook I still remember the day I was working at Apollo computer and found out that our workstations were being used in Air Traffic Control. I was horrified.
Add comment