The vast majority of the US's housing stock has been paid off. Every time a residential property changes hands, the bank gets to re-collect all of their fees for...what, exactly? Making money available? They only do that because they're underwritten by the federal government, subsidized by taxes.
So why don't we just give direct loans to people, and subsidize those who need it directly instead of funneling the money through dozens of greedy hands taking percentages off the top?