It was never supposed to be confidential. That need arose as a direct result of using it as an ID. If the SSA was the only organization using the number, (as originally intended,) then it wouldn’t need to be kept confidential.
But when the SSA gave every single person a unique number, other organizations went “hmm this sure would be convenient for differentiating individuals with similar names and DOBs.” So other organizations started using it for identification, and suddenly you needed to keep the number secret because anyone with your number could ID themselves as you.
The SSA needs to publish a public database of every single name, DOB, and SSN. Force organizations to figure out a new system of identification, instead of relying on an insecure and outdated system.