Many states have allowed people who are not constitutionally eligible to be president to be on the actual November ballot while others do not (Colorado is one that enforces it at the ballot level). These have been candidate for minor 3rd parties that may not even have enough ballot access to win even if they won all the states they were on the ballot or write-in candidates for, so it hasn't mattered. But such requirements could alternatively be enforced at the electoral college level or congressional level. Iirc, the 14th amendment does not make it clear at what stage to act. I think the November ballot is a reasonable place.
Given primaries are basically private companies just planning on who they want to try to put in the federal ballot, the state not interfering seems pretty reasonable to me.