If you want to support for foreign languages/loanwords or add in symbols to be combined with text (very useful for visual novels, websites, etc.) then the Unicode Code Charts are basically a recipe book for what you can include, what it roughly looks like and where it goes to work in a font.
The Unicode Proposal Documents include great info on historical punctuation and letters if historical styles are of interest.
So is MUFI (the Medieval Unicode Font Initiative).