The word "alone" comes from a compound of "all" + "one"

Yep! Surprising but true!

Alone is first attested as Middle English allone, and earlier as two words, all oon.

Lone comes from a later shortening of alone (Apheresis is the technical term).

Alone and one don’t rhyme anymore because of the irregular change from /oːn/ to /wʊn/ in Late Middle to Early Modern English. See also only, which comes from one + -ly.