If I got with a fat girl who wanted to lose weight and I made her feel bad about it because “then you won’t be hot anymore” that would be kinda shitty right?
Yes, on the grounds that it would be shitty to intentionally make someone feel bad. However, if after the woman lost weight you felt less physical attraction to her, I would understand. And then it’d be up to you to conclude at that point the importance to you, and if it’s fair to either of you to continue the relationship. Surely there’d be some moral debate if ending a relationship on those grounds is “right”, but that’s really not for anyone else to decide.