Let's explore why a system capable of partial driving automation (like FSD Beta) and automated driving systems more broadly are decidedly not at all like ChatGPT.
Here we have another #Tesla#FSDBeta clip showing some highly-questionable automated vehicle behavior (the automated vehicle proceeds through a marked crosswalk where a pedestrian has already entered).
Let's put our #SystemsSafety caps on and take a look here...
@brennansv I have a thread coming up on that soon - on the issues with this #NHTSA data collection program (which I might combine with a criticism of #NCAP “five star” ratings programs).
In short, it is an effectively worthless program given that the “data” collected is unaudited.
Ok. So recently, #Tesla had published an "Impact Report" which contained a slide presenting some "data" that their #Autopilot and #FSDBeta products "enhance safety".
And one article and one Twitter thread caught my eye in scrutinizing these numbers.
While the analyses and arguments in this article and thread are not necessarily wrong, there are more fundamental issues here that need to be surfaced in my view... so let's take a look.
@enmodo He does. Repeatedly. Twice just this month by my count.
It has all the hallmarks of an individual that simply cannot understand or appreciate the actual safety issues.
The press seems to catch him in between tech conferences and I have zero clue how the Secretary of Transportation, with serious and ongoing roadway and rail safety issues no less, has time for any of that.
The #NHTSA is without permanent leadership today, as we speak.
@enmodo I really did not say much of anything there.
I meant "outsized" in terms of a constellation of historic (like increasing vehicle size) and emerging (like automated driving systems) vehicle safety issues that are growing and converging.
And, no, the dysfunction and obstruction in #Congress is not a valid excuse in my view.
I do not recall an open letter with thousands of prominent signatories, a hastily-assembled White House Task Force and a big Senate hearing for automated driving systems.
You know... #SafetyCritical systems that are masquerading as #AI that have killed people and have the capacity to readily cause immediate injury and death.
We have an unregulated Wild West out there on that.
@Craktok The #NHTSA, the federal car safety regulator, actually has some pretty broad powers, in theory.
But decades of subservience to automakers have eroded their independence, competence and credibility.
The NHTSA has only had a whole 3 months with a permanent administrator during #Biden’s term.
The White House could have made some powerful moves, without Congress, to significantly rein in this Wild West… but they have not, across three different administrations now.
Alright, so last week, a jury in California rejected the claims brought against #Tesla's #Autopilot product.
I am not a lawyer and the detailed analysis of legal issues is orthogonal to the obligations of #SystemsSafety experts in educating the public and in criticizing regulators.
The responses from the jurors in this case are interesting, though... and very much expected.
This case, this "bellwether" case that is being celebrated by some, should have never made it to a jury.
It did make it to a jury because the #NHTSA has so completely dropped the ball on even marginally regulating the marketing and design of these extremely dangerous and inherently deceptive systems - systems that, by their very definition, are structurally unsafe.
An interesting article here by @mimsical and I would recommend reading it.
I think it is a reasonable take on how, essentially, the regulatory landscape will look in the US and perhaps elsewhere.
That said, I have some notes.
Not so much on the article itself... but on my favorite punching bag, the #NHTSA.
For those that do not know, the NHTSA is the unserious, disinterested and effectively theoretical regulator in the US for vehicle and roadway safety. 🧵
@mimsical But the immense complexity of the roadways are a given and they should be a given to #ADS developers and fleets.
System developers and fleets cannot unilaterally change that.
There is no technical value in trying to fix "fault" on any roadway participant (as the #NHTSA does, which is partly why the US #RoadwaySafety record is abysmal).
The obligations of systems developers are to validate their #SafetyCritical system in a selected physical environment - with its warts and all.
@mimsical Again, the #NHTSA is not remotely capable of that because, for decades, the NHTSA has hidden behind a "blame the human" dogma and shallow #NCAP assessments (often known as "five-star vehicle ratings").
Focusing on "the math" (an endpoint) with respect to #ADS safety would therefore be familiar ground for the #NHTSA and I do expect them to embrace it - as this article predicts.
But the unique risk with ADS safety is that "the human" can no longer be readily blamed anymore.
@mimsical Let me just say that I have never heard a remotely convincing argument yet as to why #ADS cannot be regulated with almost the exact same processes that commercial aircraft are.
That is not necessarily to disagree with this take from #Mobileye, but it is a higher level observation of mine.
No, the #NHTSA is not capable of that type of rigorous oversight (from a skill sets perspective, at the very least), but that fact does not invalidate how the regulatory processes should be.
So, Chair Khan, are we going to finally do something about manufacturers dangerously exaggerating the capabilities of #AI-equipped automated driving systems?
Or nah?
Because this has been going on for nearly 10 years now.
You know... one almost has to admire the absurdity of how a goddamn #chatbot finally blew the top off of primordial safety issues that have long-existed in the "AI" industry.
It took the #Biden Administration nearly 2 days to trot out Secretary #Buttigieg to issue a half-hearted statement defending the appointment of Professor Cummings to the #NHTSA role.
But by then it was too late.
Professor Cummings was forced off of Twitter.
It was later reported that Professor Cummings agreed, at the request of the NHTSA, to recuse herself of all matters related to #Tesla (the top industry problem) and she unceremoniously left the agency at the end of last year.
Speaking about those who tried to warn regulators about #Musk's "AI" wrongdoings and were dismissed and attacked, I would be remiss not to mention Professor Missy Cummings - one of the top minds in #SystemsSafety and automated driving systems and a hero fighter pilot.
In October 2021, the #NHTSA (the highway and vehicle safety regulator in the US) tapped Professor Cummings as a Senior Safety Advisor.
It is like Groundhog Day, once again, at the US Department of Transportation.
Incredible.
Here we are again, Secretary #Buttigieg all but admitting that #Tesla's marketing strategies are deceptive and dangerous... but not leveraging the broad powers of the #NHTSA to do anything about it.
This has only been going on for nearly 10 years now...
good read on the outcomes of the #starship#spacex launch. It really does not bode well for starship, and thats only based on the current issues with the launch. The article does not go into the regulatory risk that comes with any revision to the launch base, which obviously has to happen. It does mention that barely anybody knows the actual financials of spacex, which is also a big red flag. ZIRP is over, and financing will get more expensive.
Sure the idea is to quickly reiterate on the design, but how many reiterations can spacex still afford?
@flowerpot@laurenshof The #FAA is a deeply flawed regulatory agency that has seen its robustness and, effectively, its mandate (that was always contradictory) eroded over the years due to a variety of forces.
And that is saying something because the FAA is one of the most robust regulatory agencies under USDOT.
For example, the #NHTSA, which is supposed to regulate highway and vehicle safety is so ineffective that it might as well not even exist.
This is really more of a story about #OSHA and its immense effectiveness failures - a long-standing problem that underpins US construction and #manufacturing activities.
The agency existing, on paper, while doing nothing actually harms the public as the public thinks there is someone looking out for their interests when there is not.
It is just like the lack of effective auto regulation in the US (supposed to be maintained by yet another ineffective US regulator - the #NHTSA) and how #Tesla exploits that via its #Autopilot and #FSDBeta programs.
It is just like #NCAP testing eclipsing what should be proper auto regulation via validation process oversight.
Regulatory “illusions” designed to trick the public into thinking there is a cop on the beat.
How much money do you think Cruise/Waymo etc have thrown at London Breed and other SF gov't officials? That these vehicles are still allowed to be on public roads is some incredible regulatory capture.
@chema@mattmangels It’s that and it’s really just a continuation of effectively having zero automotive regulations in the US on a federal level.
The #NHTSA, the federal agency responsible for highway and vehicle safety, has spend decades and countless resources to construct a public image of robust regulation, while the truth is entirely the opposite.
@kentindell The real question, in my view, is if this individual even recognizes that this vehicle behavior was safety-deficient.
What we are dealing with here is "stage 2" of #Tesla's years-long, unchecked marketing campaign of exaggerating the capabilities of their automated driving systems - that is, human drivers are now just "making up their own rules" in order to advocate for Tesla (the company) and for #FSDBeta.
Meanwhile, the #NHTSA is still hopelessly stuck on "stage 1".