I doubt that’d be the reason (for iphones at least) because when you turn on speakerphone, the sounds comes out the bottom and the voice goes in the FaceTime mic at the top of the screen (beside the earpiece).
So if they want to be heard as clearly as they can, they should hold the display facing them (like you’re in a video call but no image)