@molly0xfff It's possible by setting a unicode character (fallback) in the CSS content, then style it with a custom webfont (e.g. convert the SVG with IcoMoon).
That's how I manage such link indicators in my personal website, although I move them to the right gutter instead of inline. Example: https://heracl.es/geary/
The difference between versus an SVG is negligible. Of course, opinions may vary between what's good practice between icon fonts and SVGs.