As a counterpoint to maybe help you see how it looks “from the outside” is that every time someone says sorry to me I have to stop my current thought process to evaluate what you are asking forgiveness for, how much it did impact me and if I am willing to forgive.
I would become a bit frustrated if this would occur multiple times over one interaction mainly because it would take so much energy to go through the whole process in my head.
This does not help you in your current situation but might give you some insight into other peoples reaction and why they seem to get annoyed at “sorry”