Hear me out: A scripting language that compiles to bash or sh (any suggestions?)

why?

Because bash feels clunky to write and work with for anything non-trivial, especially compared to other scripting languages.

Why not another scripting language (no compile necessary)?

Because bash and sh are installed nearly everywhere. Any other scripting language means the user is required to have that installed, and that is far less likely to be the case.

If I could write my scripts in a nice syntax, but be sure my users will be able to use it effortlessly by distributing to them compiled versions, then that would make both of our lives easier!

Thoughts? Are there any languges that do this?

jeffhykin, (edited )

I write a lot of bootstrapping scripts, and I have a solution thats probably something you and others in this thread have never seen before. You can write a single script in a full/normal language, no compilation step, and it works on systems that only have bash/sh. It doesn’t compile to bash, or at least not in the way you might think/expect it to, but it should do what you want.

(guillotine because it’s a universal executor) github.com/jeff-hykin/deno-guillotine

This^ one uses Deno/JavaScript, but in principle it might be possible to do with other languages. It definitely requires some explanation, so I’ll try to give that here;

As another person said, shells are not nearly as standardized as we need them to be. Mac uses zsh, Ubuntu uses dash, neither store a posix bash exectuable in the same place, and both have ls and grep differences that are big enough to crash common scripts. Even if you’re super strict on POSIX compliance, common things will still break if you write a big script (or trying to compile a big program to bash).

I hate JS as much as the next guy, but it is possible to write a single text file that is valid bash/dash/zsh/powershell and valid JavaScript all at the same time. It sounds impossible, but there is enough overlapping syntax that actually any javascript program can be converted into a valid bash script without mangling the JS code. It might be possible to do for python as well.

POSIX is good enough for making a small, carefully-crafted well-tested OS-detecting caveat-handling script. So that’s exactly what we do; use a small shell script at the top to ensure that the JS runtime you want is installed (auto install if missing). Then the script executes itself again using the JS runtime. It wasn’t easy but I a made a library that explains how it’s possible and gives a cli tool that automates it for the Deno runtime (the link I posted above).

After that, I just recreated tools that feel like bash, but this time they are actually cross platform. Ex:


<span style="font-weight:bold;color:#a71d5d;">let </span><span style="color:#323232;">argWithSpaces </span><span style="font-weight:bold;color:#a71d5d;">= </span><span style="color:#183691;">"some thing"
</span><span style="color:#323232;">run</span><span style="color:#183691;">`echo hello </span><span style="color:#323232;">${argWithSpaces}</span><span style="color:#183691;">`
</span>

I picked Deno because it auto installs libraries (imports directly from URL so users don’t have to install anything)

Natanael,

Can this thing run complied WASM? Because compilers to WASM from other languages exists already

jeffhykin,

It can run compiled wasm! So you could write a bootstrap script in rust, compile it to wasm, embed that wasm into a deno installer script using deno.land/x/binaryify and then ship that as a universal executor.

keegomatic,
keegomatic avatar

Okay at first I was pretty convinced that this was just the wrong way to accomplish what I thought your goal was. But now, after reading the StackOverflow post and your README, I think this is fascinating and frankly really awesome. What a clever and strange thing, using multiline comments that way, and string no-ops. I think just knowing this exists will cause me to find reason to use it.

aport,

its possible to write a single text file that is valid bash/dash/zsh/powershell and valid JavaScript all at the same time. It sounds impossible, but there is enough overlapping syntax that actually any javascript program can be converted into a valid bash script without mangling the JS code.

I’m both impressed and horrified

UlrikHD,
@UlrikHD@programming.dev avatar

It’d honestly the funniest thing I’ve read on this instance. Puts programmer humour to shame. Love it when developers finds the jankiest/unconventional way to solve problems.

Mikina,

It’s one of those tools that can both be used on a resume or as a diagnosis. I love it!

jeffhykin,

Agreed hahaha. I thought I’d enjoy the day my code golf skills would be used to solve a legit problem but instead it just feels kinda gross 😆

Honestly it’s really dissapointing we don’t just have an agreed-upon universal pre-installed language. And it’s beyond ironic (more like the universe is laughing at us) that JS, the web language that gets used for every not-web-thing, is also the language with a syntax that allows it to become the effectively universal no-preinstall language.

onlinepersona,

Bro, make a video and put it up on peertube please then link it in the README. I need to see this shit in action. It sounds awesome, but it’s 10am and my eyes are just opening, so reading through everything and testing it isn’t happening on my phone rn.

jeffhykin,

Actually I’ve been thinking of starting a Youtube/Peertube channel for a while so this will be a good place for me to start!

I’ll come back and post a response once I’ve uploaded it! It’ll probably take a week or two.

onlinepersona,

💖 looking forward to it!

snowe,
@snowe@programming.dev avatar

Not only do you solve the problem op is trying to solve, but you also made the most horrifying and hilariously ingenious thing at the same time.

Natanael,

Polyglot runtime lmao

Mikina,

Do I understand it right that what the tool does is include install scripts in all of the other languages, that simply download a portable Deno runtime and then run the rest of the file (which is the original Javascript code) as Javascript?

So, you basically still have an install step, but it was just automated to work cross-platform though what’s basically a polyglot install script. Meaning that this could probably be done with almost any other language, assuming it has a portable runtime - such as portable python and similar, is that correct?

jeffhykin, (edited )

Almost, but you bring up an important point about other language support.

The code includes an install script for one language, and the second part about “any language” isn’t quite right. There is an alternative way to get any-language support but the current approach requires a language to have a syntax that is compatible with bash/powershell. For example I abuse the hell out of multi-line strings and multi line comments in javascript to make it be interpreted as a do-nothing bash/powershell script.

Python specifically might be possible because of its triple-quote strings, I haven’t spent a long time trying but I did try a bit. However in general I don’t think languages, like Haskell or Elixr, can work in this form because their syntax is incompatible.

However, if you don’t care about being able to edit the script, it should be possible to mangle code from other languages, like converting Haskell code to hex or some other escaped format (can’t be binary because that’s not valid bash/powershell). We’d need to handle unpacking that hex with shell/powershell, but it could be done. And in that case, yes it would work with any portable language. (And many are more portable than Deno, which struggles to run on old stuff like Ubuntu 16.04!)

If you’re interested in the hex unpacking let me know. I’m working on an offline bootstrapping script for deno, which involves embedding the runtime binaries of all OS’s as hex into the script itself. Once I make it, it should be a lot easier to get this kind of thing working for other portable runtimes.

Pencilnoob,

This is absolutely cursed and brilliant. Bravo!

Now I just need to find a place to use it.

yogsototh,
@yogsototh@programming.dev avatar

I use this nice trick to use Clojure has a bash script. This auto-download clojure so this id quite portable and reproductible.

gist.github.com/…/6bb4562c4bc578ef223182e3bb1e72c…

Previously I also used Haskell’s turtle lib that could run with a portable shebang and it could even be compiled later if you need more speed.

swordsmanluke,

I’m gonna pile in with yet another option that isn’t a language that compiles to bash…

Consider Ruby for easier shell scripting. With its back tick syntax for executing shell commands, it’s quick to use when you want to glue together a series of commands. …and then you get to use a sane syntax for your script’s logic.

Ex:


<span style="color:#323232;"># Check my history for all usages of the xsv command
</span><span style="color:#323232;"># and extract the filename
</span><span style="color:#323232;">csv_files = `grep ~/.zsh_history "xsv" | awk '{print $3}'`.
</span><span style="color:#323232;">                     split("n").
</span><span style="color:#323232;">                     map(&amp;:chomp)
</span><span style="color:#323232;">
</span><span style="color:#323232;"># list any csv with my phone number
</span><span style="color:#323232;">csv_files.select { | filename|
</span><span style="color:#323232;">  `grep #{filename} "555-1234"`.chomp != ""
</span><span style="color:#323232;">}.each {|filename| puts filename}
</span>
lysdexic,

Why not another scripting language (no compile necessary)? (…) If I could write my scripts in a nice syntax, but be sure my users will be able to use it effortlessly by distributing to them compiled versions, then that would make both of our lives easier!

Would it make sense to describe your idea as a typescript for bash or sh?

monobot,

This thread reminds me of stackoverflow, most people are just convincing you in something else and it is obvious they have never been in your (and mine) situation.

Just answer question if you have some idea, yes we know python exist, that’s nice, but not an answer to this question.

ndotb,
@ndotb@programming.dev avatar

Knock off the childish fucking gatekeeping and go back to reddit. It’s what the wider industry uses.

Pyroglyph,
@Pyroglyph@lemmy.world avatar

It’s perfectly acceptable to not want to use a certain tool.

You would be the kind of person on SO to reply to a question saying “how do I do A” with “nobody does A, do B instead”. That’s not constructive.

BatmanAoD,

It absolutely can be constructive. The reason people respond that way on SO is because it is genuinely common for people to think they need something because they’ve misinterpreted the core problem they have.

Natanael,

IMHO the closest real existing thing is compilers to WASM and then using a JavaScript engine, possibly a JavaScript engine which can run in bash (like that ridiculous hack below, lmao)

AI_toothbrush,

Giga brain move: make a bash script that compiles some c code then runs it

nxdefiant,

Still requires the compiler though.

I don’t have a good answer, but I’ve embedded python in bash scripts before after having the bash script install python. Take that as you will.

sim642,

Why not another scripting language (no compile necessary)?

But you’re describing compiling that new language to bash…

cyclohexane,

Yes, I’m answering why I am not taking a completely other scripting language

In other words, I am making the case for a compiled language by answering the question of why i am not considering a language that doesn’t need it.

BautAufWasEuchAufbaut,
@BautAufWasEuchAufbaut@lemmy.blahaj.zone avatar

Is transpiling a form of compiling?

sim642,

Yes, that’s where it’s name comes from!

DrDeadCrash,

Not exactly what you asked for, but take a look at Nushell

cyclohexane,

I’m familiar with NuShell and looks very nice. But unfortunately yeah, not what I’m looking for. It would require installation by user.

maxbossing,

Install nushell vie bash and then pipe your commands into it?

vrighter,

op asked about using bash because it’s already there. Your answer is the complete opposite of that. twice

waspentalive,

And when it is powerful enough you can make it self hosting.

mundane,

Why not use a compiled language that compiles to fat binaries (rust, go etc)?

Dark_Arc,
@Dark_Arc@social.packetloss.gg avatar

It’s worth noting any compiled language can make a “fat” binary (e.g., C++), you just need to use static linking.

ShittyRedditWasBetter,

Just use Python. If you absolutely need it on every system use go and compile it down to a binary.

railsdev,

Hate this because most bootstrapping I do is within a Dockerfile where I don’t want to install the entire runtime and its dependencies just to complete a build step.

vzq,

The way I used to solve this problem back in the day was with a statically compiled Perl binary.

theherk,

Although it doesn’t crosspile to bash, I think a good middle ground is bitfield/script. Basically you can do many things you would normally script very simply with nice syntax and distribute a binary.

cobra89,

There’s always Golang which compiles to an x86 binary. But that’s probably not what you want since you probably want something readable and editable on the end system.

dack,

I don’t see why bash would be used at all here. If you want something that doesn’t need another interpreter, then just compile a binary.

cyclohexane,

Possible use case: scripts that are found in a codebase for doing… Codebase things. Like setting up dev environment for example.

match,
@match@pawb.social avatar

That’s a use case for Docker

dack,

But then building it still requires whatever scripting tool you use. Including the bash-ified version would not for practice, as it wouldn’t be very human readable and would have to be kept in sync with the source script. It’s much cleaner and simpler to just require python for your build environment.

anzo,

look into devenv.sh for that constrained use-case

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • programming@programming.dev
  • mdbf
  • DreamBathrooms
  • thenastyranch
  • ngwrru68w68
  • tacticalgear
  • magazineikmin
  • Youngstown
  • khanakhh
  • rosin
  • slotface
  • InstantRegret
  • everett
  • kavyap
  • cubers
  • megavids
  • GTA5RPClips
  • cisconetworking
  • modclub
  • ethstaker
  • osvaldo12
  • Durango
  • normalnudes
  • tester
  • provamag3
  • Leos
  • anitta
  • JUstTest
  • lostlight
  • All magazines
    •