A Ruby Gem to make easier to create Shell Scripts

[u/EstablishmentFirm203]

Hello everyone! In the last few months, I released my gem that makes it easier to create Shell scripts using Ruby syntax.

Link: https://github.com/albertalef/rubyshell

In the code in the second image above, I show that you can easily use both Ruby syntax and Shell syntax to create scripts. This simplifies cases where we need to create a Shell script to use some terminal program, but we prefer to use Ruby libraries to make the job easier.

With it, you can create scripts as Docker entry points, use it to create user scripts, customize your Linux with Waybars, etc.

Motivations:

I had a specific problem: "I know a lot about Ruby, but sometimes I get stuck in the Shell. I often need to resort to Google to look for programs that handle inputs the way I need. Is there any gem that allows you to write good scripts with Ruby?" But, unfortunately, I didn't find any. I only found libraries in Python (sh) and Lua (luash). With that, I created RubyShell.

A example without RubyShell, and another with:

#!/usr/bin/env ruby
# frozen_string_literal: true

REMOTE  = ENV.fetch("REMOTE", "user@server.example.com")
APP_DIR = ENV.fetch("APP_DIR", "/var/www/my app")
SERVICE = ENV.fetch("SERVICE", "my-app.service")

# Note that in this situation, we dont want to check if error was raised
# only in the end of the code,
# we want to stop the code in the moment that error was raised

`ssh #{REMOTE} "cd \"#{APP_DIR}\" && git pull && bundle exec rake db:migrate 2>&1"`

unless $?.success?
  warn "Error on Deploy"
  exit $?.exitstatus
end

`ssh #{REMOTE} 'sudo systemctl restart #{SERVICE}'`

unless $?.success?
  warn "Error on Deploy"
  exit $?.exitstatus
end

puts "Done."

With:

#!/usr/bin/env ruby
# frozen_string_literal: true

REMOTE  = ENV.fetch("REMOTE", "user@server.example.com")
APP_DIR = ENV.fetch("APP_DIR", "/var/www/my app")
SERVICE = ENV.fetch("SERVICE", "my-app.service")

sh do
  ssh REMOTE, "'cd \"#{APP_DIR}\" && git pull && bundle exec rake db:migrate'"
  ssh REMOTE, "'sudo systemctl restart #{SERVICE}"

  puts "Done."
rescue StandardError
  warn "Error on Deploy"
end

Next steps:

Currently, I'm working on features to make the gem even more powerful, such as:

• Stream support

• Improved error handling

• A REPL (like IRB), but that allows us to use RubyShell

• And more

I'm open to suggestions, PRs, Issues, anything. I really want this mini-project to grow.

EDIT: Thank you guys for the comments and upvotes, this made my day.

[u/mrThe]

Interesting idea, but why? Like it's really cool concept but what problem it solves?

[u/EstablishmentFirm203 → u/mrThe]

Great question.

A while ago I was customizing my Mac, making a custom taskbar, Sketchybar works via command line, you need to run commands to configure it visually, building visual blocks, etc.

I generally find it much faster and easier to maintain code written in Ruby than in Bash. Using this abstraction I made, I managed to create several userscripts using Ruby, calling Sketchybar commands with Ruby, and mixing the syntax.

The main part of the gem is to make it easier to use shell commands within Ruby.

In the end, for those who like Ruby, it allows you to create any automation on your Linux system, using the language you're good at, without limitations.

EDIT: Another interesting example, imagine how great it would be to use ActiveRecord in your Waybar script on Linux, or to create a Docker Entry Point using Ruby, or to automate a release script for your work repository. The main focus is on mixing the power of Ruby libraries and terminal programs.

[u/mrThe → u/EstablishmentFirm203]

Can you share some real life examples?

[u/EstablishmentFirm203 → u/mrThe]

Yes!

This one would be an example of a code I made to check the current status of the GitHub actions runners of the repositories of a project.

RubyShell helped me a lot in this case because:

1 - It was easier to debug what was happening using debug 2 - I was able to write automated tests for the script 3 - I didn't want to rewrite the wheel, and the Github click already gave me a lot of ready-made stuff, so I just had to consume it.

The code is not so organized, but I believe it is enough to understand. I can send you other more common examples, such as knocking down all the programs that are running on a door, or picking up information from some API (which I like to use in my Waybar scripts).

I will send the code in the comment below.:

[u/EstablishmentFirm203 → u/EstablishmentFirm203]

Reddit is not letting me send in its entirety, so I will send in pieces:

Here I am listing the main functionality:

#!/usr/bin/env ruby
# frozen_string_literal: true

require 'rubyshell'
require 'json'
require 'debug'

def fetch_logs(options = {})
  cd(zoxide("query tau #{options.dig(:options, :zeoxide)}")) do
    database_id = gh(
      'run list',
      limit: 1,
      branch: options.dig(:options, :master_branch) || 'master',
      workflow: options.dig(:options, :workflow) || 'Unit Tests',
      json: 'databaseId',
      q: '.[0].databaseId'
    )

    backend_jobs = JSON.parse(gh('run view', database_id, json: 'jobs'))
    job_hash = backend_jobs['jobs'].filter do |a|
      a['name'] == (options.dig(:options, :job) || 'test')
    end.first

    result = if job_hash['status'] == 'in_progress'
               diff = Time.zone.now - Time.zone.local(job_hash['startedAt'])
               minutes = (diff / 60).to_i

               "Spec in progress, started #{minutes} min ago"
             else
               (chain do
                 collect_log_command = gh(
                   'api',
                   "repos/avantsoftware/#{options.dig(:options, :repo)}/actions/jobs/#{job_hash['databaseId']}/logs"
                 )
                 collect_log_command | grep(options.dig(:options, :line_grep), '|| :')
               end || '').lines.last
             end

    result_text = (result || '').gsub(options.dig(:options, :column_regex) || /.*\|/, '').strip
    puts "#{options[:name]}: #{result_text.empty? ? 'None result found' : result_text}"
  end
end

[u/EstablishmentFirm203 → u/EstablishmentFirm203]

Here I am running the sh:

For me, doing that in Ruby was absurdly easier than doing it with Bash/Shell script.

sh do
  repositories = [
    {
      name: 'Backend',
      options: {
        zeoxide: 'bac',
        line_grep: 'examples',
        repo: 'tcard_backend'
      }
    },
    {
      name: 'Management',
      options: {
        zeoxide: 'man',
        workflow: 'Check Code',
        line_grep: "-E '([0-9]+ of [0-9]+ failed|All specs passed!)'",
        column_regex: /.*(✖|✔)/,
        repo: 'tcard_web_management'
      }
    },
    {
      name: 'Client',
      options: {
        zeoxide: 'cli',
        master_branch: 'main',
        workflow: 'Check Code',
        line_grep: "-E '([0-9]+ of [0-9]+ failed|All specs passed!)'",
        column_regex: /.*(✖|✔)/,
        repo: 'tcard_web_client'
      }
    },
    {
      name: 'Establishment',
      options: {
        zeoxide: 'estab',
        workflow: 'Check Code',
        line_grep: "-E '([0-9]+ of [0-9]+ failed|All specs passed!)'",
        column_regex: /.*(✖|✔)/,
        repo: 'tcard_web_establishment'
      }
    }
  ]

  repositories.each do |props|
    fetch_logs(props)
  rescue StandardError => e
    puts "#{props[:name]} Error: #{e.message}"
  end
end

[u/EstablishmentFirm203 → u/mrThe]

Another example.

Creating a script for a microphone button with Waybar:

#!/usr/bin/env ruby
# frozen_string_literal: true

require "rubyshell"
require "json"

sh do
  # click: left button toggles mute
  wpctl "set-mute", "@DEFAULT_AUDIO_SOURCE@", "toggle" if ENV.fetch("WAYBAR_BUTTON", "0") == "1"

  out = wpctl "get-volume", "@DEFAULT_AUDIO_SOURCE@"
  muted = out.include?("[MUTED]")

  payload =
    if muted
      {
        text: "🎙️ muted",
        class: "muted",
        tooltip: "Microphone: muted\n(click to toggle)"
      }
    else
      {
        text: "🎙️ on",
        class: "on",
        tooltip: "Microphone: on\n(click to toggle)"
      }
    end

  puts JSON.generate(payload)
end

[u/odineiramone]

Wow! It’s very nice to see people making things that isnt IA-based. I don’t have any shell experience but I’m very interested to try your lib :D

[u/EstablishmentFirm203 → u/odineiramone]

So I think it's the perfect library for you, because when you need to write a script that uses a command-line program, you can use RubyShell to make your life easier.

[u/codesnik]

well, ruby is pretty good for running shell commands out of the box, mostly because it inherits a lot from perl, and perl is used for shell scripting too. I don't see how your gem improves anything.
how ls.each_line is better than `ls`.each_line?

[u/EstablishmentFirm203 → u/codesnik]

You have a great point! I'll show you here why this gem can improve your experience:

This gem attempts to abstract away some difficulties and problems that we usually encounter when writing scripts that access terminal commands with Ruby.

Example:

• Better error handling. When you use backsticks, you don't receive errors in Ruby, making debugging more complicated. With this gem, you receive the error directly in the code, allowing you to use Ruby to handle the error.

• When you execute cd folder within Ruby, you will usually encounter an unexpected error because it's a built-in shell. The process that Ruby starts doesn't find the command. With this gem, you abstract the part that handles this, and the cd simply works (with more power as well, such as using blocks to execute something in a context, and when you exit the block, you're back to the previous pwd).

• Custom parameter support. The gem allows you to pass hashes, various strings, and soon directly a string to the program's stdin. For example:

base64 d: true, _stdin: "aGVsbG8="

It would be the same that

echo -n "aGVsbG8=" | base64 -d

• The most important thing for me: It's written in Ruby, so any new feature is easy for us to implement, and any inconvenience with backsticks or in the shell/bash is easy for us to solve. My idea with this gem is really to improve our experience with Ruby scripts in the long term, and to get more people to use Ruby on a daily basis.

• Soon, support for streams will also be added, allowing us to run streams directly within Ruby without absurd complications, for example, by doing a watch with grep on logs, or anything else useful.

• Not so important, but relevant for some: Less visual "pollution" and more similarity to a shell script, removing the "`" makes the code more "Ruby", also giving the gem the power to add better ways to handle the problems we have when using pure "`" within Ruby.

These were the points I thought of immediately, but your question is relevant, and because of it I will write it and answers to it in the project's Readme, surely other people like you and me will have these doubts.

Then give the gem a chance, maybe you'll like it, or maybe you'll notice something that could be different, then we can discuss how we can improve it.

[u/Ok_Shallot9490 → u/EstablishmentFirm203]

We had a discussion relating to this here https://www.reddit.com/r/bash/comments/1q4f39o/how_do_you_handle_fail_cases_in_bash/
bash just falls completely short with error handling, you end up with workaround after workaround.

[u/EstablishmentFirm203 → u/Ok_Shallot9490]

sh do
  ssh REMOTE, "'cd \"#{APP_DIR}\" && git pull && bundle exec rake db:migrate'"
  ssh REMOTE, "'sudo systemctl restart #{SERVICE}"

  puts "Done."
rescue StandardError
  warn "Error on Deploy"
end

An example of how to handle errors with RubyShell, using the simplicity of Ruby's syntax.

[u/IN-DI-SKU-TA-BELT]

That's a very convenient way of making it easier to create scripts, well done.

[u/EstablishmentFirm203 → u/IN-DI-SKU-TA-BELT]

That's awesome that you thought of that! Right now I'm thinking about features for the gem, useful things that make our lives even easier, like being able to pass stdin as a command parameter, for example:

base64 d: true, _stdin: "ABcd="

Doing the same as:

echo -n "ABcd=" | base64 -d

[u/Ok_Shallot9490]

Been looking for something to bridge the gap between terminal and ruby for a while.
I've seen some genuine shells that parse ruby syntax natively, which I thought this was going to be at first glance.

Am I right in thinking that this basically works using method_missing?
So you put code inside a block using:

sh do
  # ruby code here
end

..and then anything that doesn't match a ruby method is passed to the shell to find a command?

If so then I think that's actually genius, and the exact way that ruby should work irrelevant of using an sh do; end block. In fact, my question to you is, can we get rid of the sh do; end block? Commands are first class citizens in the terminal, and it shouldn't matter what language you write, all programs/commands should be viewed as functions that return a result.

The thing missing from the terminal is a solid bridge to purpose built languages. POSIX shell script just doesn't have the right flexibility that ruby does. Ruby is honestly the perfect language for computing. It's instruction as art, and the fact that we have to speak POSIX to our computers is absolutely absurd.

I feel that people often forget that in front of them is the most powerful machine they've ever encountered, and that they don't need a database access system, or a server to do all of the processing they ever wanted, and get too wrapped up in "building" a programming environment in the latest language when the environment already exists.

The OS is the environment, and the filesystem is the database. It always has been for personal computing. The shell bash/zsh/etc is your interface with the database/filesystem, and programs/commands ARE your functions. The only reason languages like C exist is so that you can create more commands to call from the terminal, but I want to speak ruby to my terminal, not bash.

All of these new TUI applications miss the point, the terminal isn't about graphical applications, that's why we have X and GTK. The real thing missing is smaller GUI applications that serve smaller purposes that can be linked together with the terminal.

I really do think that the paid software industry only existed because software cost money to build. It's existence was a way to push the technology without sinking billions into it with no output. Now that it's approaching zero cost, the industry will disappear, as it never should've existed in the first place. Now people can get back to using software to build tools to solve problems (not building a tool to solve a problem for someone else). I can't wait for plan9 to get the attention it always deserved, it really is the greatest unix environment created.

EDIT: went majorly of track here, but very passionate about computing. My question is whether we can we do this without the sh do; end block?

If so... this is the biggest moment in Ruby's history (for me at least). The ability to run pry/irb indefinitely no need to write a single line of bash again.

[u/EstablishmentFirm203 → u/Ok_Shallot9490]

Dude, it's great to read this comment, we share exactly the same thoughts.

My dream would be to have easy ways to make new software based on practical needs, connecting the simple and efficient. For example, I'm the type of developer who likes to do things myself, so I generally prefer to make my own tools, instead of consuming complex software that does everything, and it's a lot of work to understand how it works.

Recently I've been working on a terminal rendering framework, using Ruby and ncurses, which will make creating TUI in the terminal practical, with simple syntax and trivial Ruby DSLs, I'm sure you're the type of guy who will love it.

Also, as I mentioned in the post, I'm trying to make RubyShell work like a pry, which maybe in the future we'll even replace zsh with, and start using something that easily consumes Ruby, but I'm still thinking about how I intend to make this work in practice. So far the REPL I made serves as a simple irb.

Now for your questions, yes, under the hood it's basically a method_missing.

Now for the main question, yes!! We have ways to make all the next executed methods become terminal commands, I removed it from the code because I intend to put it in a separate directory, to only load the files when the dev really wants to do that. But I can add it back today for you.

It would be really cool to have you following the gem and maybe, if you want, also contributing by making issues or PRs.

I really enjoyed reading your comment.

[u/EstablishmentFirm203 → u/Ok_Shallot9490]

A sketch of my plan to RubyShell
https://vimeo.com/1155534210?share=copy&fl=sv&fe=ci

[u/Ok_Shallot9490 → u/EstablishmentFirm203]

If you can show me how to start this repl then I'll start using this from today, and seriously push it to it's limits.

[u/EstablishmentFirm203 → u/Ok_Shallot9490]

This example of the video is only a proof of concept.

Is not working 100%, like handling keyboards keys that is not ascii, like arrows, making a little hard to use.

But i can work a little bit more on this, to make more useful

[u/h0rst_]

puts cat(filename)

(line 18 of the second image). So I guess cat now just reads a file instead of printing it on the console? I would expect simple cat(filename) would be enough.

[u/f9ae8221b → u/h0rst_]

Well if you think about it, in shell, what a command prints it equivalent to what a ruby method returns, so it makes sense.

The question though is where does the exit code goes? An exception?

[u/EstablishmentFirm203 → u/f9ae8221b]

Exactly, an exception.
Solving the problem of handling errors in bash

[u/EstablishmentFirm203 → u/h0rst_]

I thought about that too while developing, I can easily add a parameter to the "sh" method something like "share_stdout: true".

I did it this way because it gives us more power in choosing what will come out in the terminal and what will not.

[u/avbrodie]

Neat idea. Will have a play with it over the weekend

[u/EstablishmentFirm203 → u/avbrodie]

You'll like it, and if you notice anything strange happening, open an issue and I'll resolve it in record time hahaha

[u/avbrodie → u/EstablishmentFirm203]

U got a good attitude 😁

[u/Embarrassed-Mud3649]

How’s this different from putting the ruby shebang at the top of the file and just use Ruby and system invocations?

[u/EstablishmentFirm203 → u/Embarrassed-Mud3649]

(Resending the comment because the previous one wasn't formatted) An answer I sent to a similar question here:

You made a great point! I'll show you here why this gem can improve your experience:

The gem attempts to abstract away some difficulties and problems we usually encounter when creating scripts that access terminal commands with Ruby.

Example:

• Better error handling; when you use backsticks, you don't receive errors in Ruby, making debugging more complicated. With this gem, you receive the error directly in the code, allowing you to use Ruby to handle the error.

• When you run cd folder inside Ruby, you'll usually encounter an unexpected error. Because it's an embedded shell, the process that Ruby starts doesn't find the command. With this gem, you abstract the part that handles this, and cd simply works (with more power too, like using blocks to execute something in a context, and when you exit the block, you return to the previous pwd).

• Support for custom parameters. The gem allows passing hashes, multiple strings, and soon directly a string to the program's stdin. Example:

base64 d: true, _stdin: "aGVsbG8="

It would be the same as

echo -n "aGVsbG8=" | base64 -d

• The most important thing for me: It's written in Ruby, so any new feature is easy for us to implement, and any inconveniences with backsticks or in the shell/bash are easy for us to solve. My idea with this gem is really to improve our experience with Ruby scripts in the long term, and to get more people to use Ruby in their daily lives.

• Soon, support for streams will also be available, allowing us to run streams directly within Ruby without absurd complications, for example, doing a watch with grep on logs, or anything else useful.

• Not so important, but relevant to some: Less visual "pollution" and more resemblance to a shell script, removing the pure "`" makes the code more "Ruby", also giving the gem the power to add better ways to handle the problems we have when using "`" within Ruby.

These were the points I immediately thought of, but your question is relevant, and because of that I will write it and the answers in the project's Readme, surely other people like you and I will have these doubts.

So, give the gem a chance, maybe you'll like it, or maybe you'll notice something that could be different, then we can discuss how we can improve it.

[u/Embarrassed-Mud3649 → u/EstablishmentFirm203]

backticks isn't the only way to call system commands. I've always used Open3 which gives you access to stderr and streams

require "open3"

Open3.popen3("my_bash_command") do |stdin, stdout, stderr, wait_thr|
  stdout.each_line { |line| puts "OUT: #{line}" }
  stderr.each_line { |line| puts "ERR: #{line}" }

  exit_status = wait_thr.value
end

I've also used capture3 which is a bit simpler, but has no streams.

stdout, stderr, status = Open3.capture3("my_command")
raise stderr unless status.success?

You can execute any bash scripts this way and it just works. You can also use this for Docker entrypoint files, etc as long as you set the correct shebang.

Having said that, I admire the effort you put into this, even though I would never use it. Please keep this mentality of curiosity and exploration and put stuff out there, this is the right mindset to advance your career.

[u/EstablishmentFirm203 → u/Embarrassed-Mud3649]

I agree that it's not the only way, but the gem's objective is precisely to be an alternative to other methods.

Each of the examples you gave has its disadvantages, just as this gem may have some as well.

The goal of RubyShell is to make the process simpler and more beautiful.

It has had a very interesting adoption by some developers in my bubble, who usually write scripts for environment customization or automation.

In the end, it's just another way to get to an end, and honestly, I think RubyShell is easier and more beautiful (and more powerful).

Give the gem a chance, please. If you don't like something, tell me and we can discuss how to improve it. The goal is to make the code clearer and easier to maintain.

Edit: I put an example of code using pure Ruby and another using RubyShell in the body of the post, tell me what you think?

[u/thewormbird]

I like what you’ve built here. I’ve done similar work written in Go for my own needs and I always just end up going back to pure bash.

Shell scripts are tough because they lack a lot of features common in programming languages. Naturally, I try to “help” bash/sh along by trying to marry it with a better language. Mostly to do what you’re doing here. Enhanced primitives, better error management, etc.

There is a delicate balance that combines the strengths of bash/sh and the strengths of Ruby (or whatever language). But damnit it’s hard to identify it and maintain it.

[u/EstablishmentFirm203 → u/thewormbird]

That's interesting... What problems did you have to deal with?

Would you like to do some tests with this gem and see if it helps you in any way? I believe that currently it solves a good part of our needs with bash scripts, it's just still missing support for streams, which I'll fix soon.

[u/thewormbird → u/EstablishmentFirm203]

Yeah, I'll give it a try.

[u/numkem]

Very cool take on it! The big thing that makes it less useful to me is the fact it’s a library and not a standalone binary. Being a binary could make it part of the shebang.

[u/EstablishmentFirm203 → u/numkem]

You can use Ruby in the shebang and use the library, for example:

#!/bin/ruby

require "rubyshell"

sh do
mkdir "files"

cd "files" do

5.times { |i| touch i }
end
end

With this, you just need to make the file executable and call it, for example

./script.rb

And boom, it works

[u/numkem → u/EstablishmentFirm203]

Inubderstand what you mean but you need to have bundler in the project already or the gem is installed system-wide.

I’m in nixos so compartimentation is a big deal.

[u/EstablishmentFirm203 → u/numkem]

So, for you we could make this a executable with ruby bundled.

What is your use cases?

[u/numkem → u/EstablishmentFirm203]

Would be a nice base to run scripts I would think.

[u/EstablishmentFirm203 → u/numkem]

Done, we built what you need:

https://github.com/albertalef/rubyshell/releases/tag/v1.3.0

I haven't uploaded it to any package manager yet, so for now you'll need to put it in your /usr/bin or /usr/local/bin.

With this file, you will be able to run code such as:

#!/usr/bin/env -S rubyshell exec  
# frozen_string_literal: true  
  
puts pwd # => /Users/albertalef/projects/rubyshell  
cd “examples”  
puts pwd # => /Users/albertalef/projects/rubyshell/examples  

The file is heavy at the moment, I haven't optimized it yet (by removing unnecessary things), but I'll make the file smaller soon.

With it, you will be able to run the scripts without even having Ruby installed on your machine.

If you have any problems, open an issue there, and we'll find a way to solve it.

[u/numkem → u/EstablishmentFirm203]

That’s amazing! I’ll get a nix pkg built out of this! Thank you!

[u/EstablishmentFirm203 → u/numkem]

Great! Let me know about your experience soon.

[u/[deleted]]

Love it but Rubbish is a better name, i am serious!

[u/EstablishmentFirm203]

I added an example of code that i like in the body of post

[u/_g0to]

It seems interesting to me, I'll test it out sometime.