Not necessarily. You can use any server/pidgeon to send your message while your contact uses a different server to send. Also you can at any time change which servers you use and it is planned that the servers get rotated automatically in the future. There is no point in time where one pidgeon is responsible for multiple connections, you are using a bunch of pidgeons and swap them out all the time.