I would not say that necessarily. I asked the Gentoo people and they had similar setups with 12h. At 4GB ram it’ll mainly come down to swapping, so disk speed.
If the 5 days were you experience, was it using a slow hdd?
Edit: remember to disable pgo, wherever you see it. It can double your compile times