The phrase “diversity hire” implies that they were not hired for their abilities. It attacks the person looking for a job rather than one making the hiring decision. So yes, it is a slur.
The purpose of phrases like “diversity hire” is to dehumanize and marginalize without ever having to learn anything about the person you don’t like and you won’t bring about inclusion by embracing those tactics, but it is noticed and called out, but it is done as a conversation and against the individual doing the hiring, not against a group of people who were hired and did nothing wrong.