A Fortune 500 company hired me as an expert consultant to help them find out why their employees were not trusting their Explainable AI (XAI) system. My solution that worked really upset the VP of Engineering.
He was angry because my solution didn't involve any substantial algorithmic changes. During the presentation, he said, "So where are the model changes in all of this?"
Me: Why does every solution to every problem have to be a "technical" solution? Your problem was a sociotechnical one. The reason why you couldn't solve it in-house was because you were trying to solve the wrong problem.
Him: But this is such a "simple" fix? Where is the complexity in this?
Me: Why do you want to use an 18-wheeler to haul your groceries when you can do it with a bicycle? Elegance and simplicity are virtues, not problems.
Sensing the VP getting more aggravated, a senior engineer jumped in: Actually, thanks to Upol's solution, we saved a lot of money and effort on not having to overhaul the entire thing. The simplicity makes it scalable and model agnostic. We implemented everything while keeping the system live.
My main takeaway? The indoctrination of some people about (X)AI can be so algorithm-centric that they can't literally think outside of the (black) box. It speaks volumes about the gaps in their training-- every single problem in the real-world is a sociotechnical one; yet, almost every impulse they have is to solve everything using technical means.
Universities need to revamp how they teach AI-- focus on sociotechnical elements as much as the technical ones
Companies need to recruit multi-disciplinary folks like psychologists, sociologists, and anthropologists in their core technical teams
These people should be first-class citizens in the teams and be paid similar to engineers because the value of their work is just as important as the people who primarily code.
Tech educators and industry end up producing smug, self-satisfied fascists who are all quantitative without the qualitative in knowledge and are severely lacking in comprehending that others have rights which they are constantly, blatently violating.
@upol That’s been offloaded to designers since day one. When an org doesn’t want to look at the unintended harm it can cause, no group within it can persuade those in power to do so.
@upol my go-to-guess when people want a technical solution and do not accept another solution is that not-technical solutions are very solution-like. That is, in organizations, a solution should be "a thing" that is pointable-at, billable, not there before. If not (like anything "sociotechnical"), it gets harder to explain to superiors what "thing" was done and why it was worth the money (even if much cheaper, actually!)
@simulo indeed, hence the need for training if these "superiors" to develop a broader lens of what counts as interventions. I was privileged in this project bc I was a subject matter expert. I can imagine others having a harder time getting the same point across.
Add comment