Anyway this seems all kinda ridiculous because it's about some potential problem when using kwallet as the secret service and kwallet configured to use #gpg as the backend? I never knew that was possible.
Anyway, currently I'm using #keepass with its secret service integration to make all this work.