I need to convert #HTML to #Markdown and I'm looking for a tool to do that.
The output should
• preserve line breaks in paragraphs
• not contain additional, unnecessary linebreaks (e.g. 4 empty lines between paragraphs)
• be configurable (e.g. whether to use * or _ for emphasis, or * vs - for unordered lists)
• if possible, allow me to hook into details (e.g. to convert <pre class="shell"> to ```sh)
#Python or #CLI. Alternatively, what's a really configurable prettifier?
@shochdoerfer yeah. "Quite quickly" is a rather relative term 😁
We have with multiple hundreds up to thousand entries a delay measuarable in seconds. Just for the conversion. The individual conversion is quite fast and for a one off solution for sure not an issue. But when you do that in sclae it suddenly has an impact... 😕
@scy I haven't looked at the configurability details of html2text, but it converts html to Markdown. Yes the one from Aaron Swartz: https://github.com/Alir3z4/html2text/
@dunkelstern Is there an easy way to "strip" the whole input document in bs4, like throw away indentation and line breaks? Because markdownify chokes on that. E.g.
<p>
foo
</p>
becomes
foo
in Markdown (with a fucking leading space lol, your Fedi client might not display it)
@scy get_text() has a strip=True parameter which removes whitespace. First parameter of that function defines how to replace linebreaks. If you want a generator you can use .stripped_strings on the tag or parser instance and will get an array of strings
• Pandoc (it doesn't allow configuring bullet or emphasis characters)
• Prettier (it doesn't allow configuring basically anything at all)
• the Python "markdownify" package (it has a lot of trouble with indentation and whitespace in the HTML)
@scy not a full solution, but if you decide to write something of your own, you could at least use pandoc for the heavy lifting of parsing the HTML, and transform it into an AST, and then use e.g. https://github.com/jgm/pandocfilters to format the AST into a Markdown representation of your liking.
But I hope someone has done that already, even though I cannot find it…
@ki Works for super simple Markdown documents, but just think about having to track whether you're in a code block (and need to keep it untouched) or not
@scy
I see... could work with awk but I get that you don't want to do that...
the library version of pandoc provides an abstract syntax tree iirc, but that requires Haskell programming. I'm out of ideas then, sorry
Add comment