No, it is not just racism. There would have been an element of that, but it’s certainly far from the main reason. That idea is contradicted by the facts that a very significant portion of Indigenous people and Indigenous activists voted against it.
Linking to this useful post, explaining why various progressive groups were against it.