Reminder: The bisexual community has long since embraced "bi" to mean "2 or more genders", not simply "Men and Women". Some people who are attracted to many genders prefer describing themselves as "bisexual", others as "pansexual" - both are valid ways to describe yourself, whether your attractions are only to binary people, or inclusive of enby and agender people.
Yes, different people within the bi and pan community will describe their sexuality in ways that show why they feel more connected to "bi" or "pan" as what they call themselves, and that's always fine ... when they're talking about themselves. Nobody gets to tell a bi person "you're ACTUALLY pan", or vice versa. Those meanings are important to people, and we should respect them.
It's enbyphobic to insist that "bisexual" excludes non-binary people - because there's a SHITLOAD of non-binary people who define themselves as bisexual. Just let people use the words that work for them, and if you're confused about what they mean to that person, you can just ask them.
