Be mindful of the fact that the PWA is effectively a browser tab running with the profile from which the PWA was installed. There is no separation or segregation of site data, browser history, etc.
Alas, the PWA cannot be installed from an Incognito browser session, however on (some?) mobile devices the site can be added as an icon or launcher from a more private browser such as Firefox Focus. This is not the same as the PWA, but will open the site in a browser tab at the click of the icon. When site data is not stored between sessions, it will be necessary to log in each session - a trivial task when using a modern password manager.