No, I mean it depends on how each nonbinary person see it.
For some people the base gender is the same, or their gender is related/derived from the binary gender they were assigned. These people may not consider themselves trans, but rather cis or any other gender modality.
There’s no “IMO that’s not the gender they were assigned at birth” when many non-binary people do, in fact, feel they still relate to their AGAB or that their gender and their AGAB are the same or similar.