AOSP and even factory kernel source tends to be only mildly useful for proper Linux phone use. Android phones tend to ship with old kernel revisions that the chip maker forked a long time ago and developed their chip drovers on without following accepted kernel conventions or submitting any code to the actual kernel maintainers for proper review and integration into the most up to date “mainline” kernel. Due to this, and the fact that phone makers need to constantly ship new products out the door, the quality of this code added into the old kernel is often garbage, poorly commented and with no documentation. Usually no git history either.
There are other teams of people trying to clean up and/or rewrite these drivers from scratch in a way that is reviewable and acceptable in mainline. Only a small handful of the vast number of phone chips have such support, so proper Linux phone is limited to a small selection of hardware. The designed-for-Linux librem and PinePhone models intentionally chose old chipsets because these chipsets had good mainline support and thus could receive actual kernel updates rather than being stuck forever on an ancient kernel release from the manufacturer that has long since been abandoned.
Lately the Qualcomm Snapdragon SDM845 chip is seeing growing mainline Linux support and quickly becoming one of the most viable chips for mobile Linux that isn’t a complete dinosaur in terms of performance and power draw. The OnePlus 6 and 6T, which both use the SDM845 chip, have become quite popular as Linux phones now despite not yet having VoLTE and thus being useless for calls. I carry a OnePlus 6T as a secondary non-phone pocket PC because the Linux experience is very good other than the lack of phone and camera functionality. It’s fast and can do all my terminal and coding stuff as well as run full fledged web browsers well.