dansup, 
The Pixelfed backend should separate Comments/Replies to it's own table/model
This won't be an easy undertaking, but it will improve performance and enable a better comments API with proper pagination
I will start this process now, as I'm in the middle of shipping the Groups feature which will also require new tables/models accordingly.
I promise it will be done properly, with no breakage
It may seem boring to most ppl, but I love technical challenges like this