Sure all privately-owned cars are problematic, but I think it’s ridiculous to suggest that small vehicles have the same impact as massive SUVs/trucks.
Bigger cars are not only more damaging in crashes with pedestrians/cyclists (which you’ve mentioned but seem to think is unimportant?!) but they also cause more accidents because of poor visibility and longer stopping distances. Also, environmentally-speaking, larger vehicles have a much bigger impact than small cars. Not only are they less fuel-efficient, they also cause more damage to roads leading to more frequent need for repairs.
I agree that ultimately reducing the total number of privately-owned vehicles should be the main goal, but we can also simultaneously strive for smaller cars for the vehicles that are on our roads.