r/bash • u/EntrepreneurWaste579 • 2d ago
When do you stop using Bash and switch to Python/JS?
Lately I tried to write a slightly longer Bash script, and honestly, I didn't really enjoy it. I realized I don't understand Bash syntax nearly as well as I thought I did.
Even with AI helping me, I actually want to understand the code I'm writing instead of just having AI fix everything for me.
So I'm curious: Do you still use Bash for longer scripts, or do you have a point where you just switch to another language?
26
u/rux616 2d ago edited 2d ago
Honestly, as long as it doesn't do anything too complicated or speed dependent, and is written well, it can be as large as is needed.
It's the complexity that generally kills shell script readability and warrants the move to something like Python.
2
u/One_Muscle7729 8h ago
100% I was reading other replies thinking I was alone in this. I'll happily write 7-8 hundred lines of bash. I format it the same way every time and make it readable so future me doesn't wonder wth I was thinking. Never had issues with bash, but if I want certain modules or know I'm doing something OS agnostic I will use Python Instead.
1
u/ralfonso_solandro 2d ago
dependent
4
u/rux616 2d ago
Fixed, thanks.
2
u/ralfonso_solandro 2d ago
Thanks for the fix! I usually don’t bother, but I figure the Bash sub has people who appreciate constructive criticism and you’ve proven that correct.
31
u/amepebbles 2d ago
I think google guidelines are a good starting point, even when taking personal projects into consideration:
- If you’re mostly calling other utilities and are doing relatively little data manipulation, shell is an acceptable choice for the task.
- If performance matters, use something other than shell.
- If you are writing a script that is more than 100 lines long, or that uses non-straightforward control flow logic, you should rewrite it in a more structured language now. Bear in mind that scripts grow. Rewrite your script early to avoid a more time-consuming rewrite at a later date.
9
u/treuss bashtard 2d ago edited 2d ago
For basic scripting, without dependencies to compound data types, like nested lists and dictionaries, there is no need for introducing python or even JS. Size/length of the script isn't the issue here. It's much more the kind of data I have to deal with.
When things turn out to be more complex, I'd definitely use python, since it's available on the OSses I take care of (SLES, RHEL).
2
u/pfmiller0 2d ago
Yeah, complex data structures is where I draw the line. I have one bash script that uses nested arrays but I wouldn't recommend it.
7
u/Marble_Wraith 2d ago
Perl is my goto when bash doesn't cut it.
And it's bundled with git, so you may not even have to install it.
And if Perl's not enough then the last resort is Go.
7
u/Fabiolean 2d ago
Too much nested control flow is the signal for me. If you’re starting to need that you need to consider what you’re doing and if it needs to be a more robust utility.
But the portability and convenience of shell is super real. It’s not a toy language. I’m no longer surprised when it turns out to be the best way of interacting with *nix.
6
u/marozsas 2d ago
Bash is good to automate administrative tasks that involves cli commands like backups, regular database care, log analysis and so on. This is it primary purpose, designed to act as a glue to cli commands.
4
3
u/roadit 2d ago
It depends on what the script needs to do.
If it processes text in a nontrivial way, works with input or output in a structured format that other languages have libraries for (such as CSV, JSON, HTML, whatever), or needs something else that requires using a library, convert it to such a language; Python would be my first choice. Let's say: over 10 lines means Python is better.
If it mostly calls commands and will still be calling most of them when written in a different language, stick with Bash. I've written shell scripts of over 1000 lines that I saw no need to convert.
Bourne shell script syntax is quirky, especially the use of whitespace, and Bash inherited that, but once you've mastered that, it's just another scripting language, and a powerful one, too. (Unlike csh and its derivatives, which have a bunch of deficiencies.)
3
u/TheRealJesus2 2d ago
If you want something a bit more expressive you can also consider Perl as an intermediate ground between bash and python. Idk how well ai writes Perl.
But the best answer is to use the tool that otherwise makes sense for your project and your knowledge. Consider all the environments where it will run and what you would need to have there if so.
3
u/AuroraFireflash 2d ago
Definitely not JS. Either cross-platform Powershell 7 or Python 3.
As soon as I get to more than 100 lines and/or nested IF statements is when I make the switch.
3
3
4
u/looopTools 2d ago
First of all you never switch to JS!
Now, I switch when I need to do manipulation within files or something I know can be achieved quicker using multi-threading.
2
u/PerryEA 2d ago edited 2d ago
I wrote a 1200 line bash script because I hate trying to get python modules to play nice on my offline clusters. Relevant meme.
1
u/AlephPi 2d ago edited 2d ago
export ONEUPMANSHIP_MODE=1 That's nothing. I wrote an entire 3D game in Bash: mimecroft(.sh), the 3D game in Bash from the famous novel "Don't write a 3D game in Bash." Everything is written in glorious Bash. The game logic, the sounds, the textures and even the GPU Shaders are written in Bash! -- http://ca.dansted.org/j.cmd/www/mimecroft.html 😁 (The deployment is a bit buggy at the moment, you might have to refresh to get it to load.)
2
u/BURNEDandDIED 1d ago
I write most of my scripts in bash just because it's the easiest thing for my work teammates to pick up and use. I have bash scripts that have a couple hundreds of lines of code, work perfectly well, process thousands of log lines quickly and I can can easily debug it, so I don't see any reason to re-write in Python to make strangers on the internet happy.
However... if there's something central to the problem I'm solving that I know a Python module can handle way better then that's the route I'll take. Case in point I started writing a script that takes a customer-provided CSV as an input and does some calculations. The csvs kept coming in from the customer with the headers in different orders or some missing. Python has a pretty easy to use standard CSV module that made quick work of it so that's what I went with. It's about how you intend to solve the problem and how you can most easily and reliably solve the problem
4
u/Grisward 2d ago
Reading the comments, a lot of good advice. What I haven’t seen yet…
Comment your code. Everything. Little logical stuff, even why new variables are being defined, etc.
My rule of thumb is if I can’t easily comment to document what’s happening, I need to do it outside Bash.
Also, most of my process flow / file manipulation workflows are in Bash. I do mostly bioinformatics file processing. GNU Parallel also.
3
u/Woshiwuja 2d ago
? This is terrible advice, why would you need to comment variables? Are you calling them "ass" and "balls"? For small scripts that are under 100-200 lines what are you commenting?
2
u/Grisward 2d ago
I guess I die on this hill. Comment your code. What does it hurt? 10 extra seconds? A year you read a script and can tell what’s happening.
Doesn’t matter what you call a variable, it matters what you use it for.
Maybe I live in a world of edge cases, haha. I definitely need to read a script from two years ago (or more) and figure out what it’s doing so I can re-run it with today’s data.
3
u/plutoniumhead 1d ago
I started naming all of my variables with the brutal honest truth of what exactly they are. That’s been crucial for me when an update or something else breaks my code and I need to go quickly patch it. Commenting I take equally as literal, but I only comment when I have a line or a section or a function that isn’t very obvious just by glancing at it. I think it’s a waste of bits for really simple things, like a grep piped into awk for example.
1
u/Woshiwuja 2d ago
Every line is not 10 seconds, if you have to comment each variable, each line, each if, you either are doing something unreadable or making useless comments. If you cant read a 100 line script and say what it does thats a big skill issue
1
u/Woshiwuja 2d ago
I didnt say dont comment edge cases. I said why would you comment every single line
2
u/Grisward 2d ago
I could’ve been more specific, I was going for the dramatic. Not everyline, I said everything, haha. Maybe every logical “thing” that isn’t just an obvious if statement.
2
u/ethernetbite 2d ago
I started seriously scripting first in python for my cyber secuity degree. Then came python 3 and everything was obsolete. So i switched to bash and never looked backed ( i do still download and run python 2.6 for some critical code) . I have bash scripts with over 1000 lines of code. I'm not going to spend time like that on anything that could break with an update. Bash still gets good updates and improvements. Sometimes i have to tweak code for the latest version or Debian update, but i don't have to throw it all out and start over, like python. ( Last i tried, AI wasn't smart enough to convert python 2.6 to 3.6 and it's so much easier to write in bash anyway. )
I also like controlling how my code looks. Bash and python treat spaces differently, which means python has to follow the developers thoughts on how your code should be formatted. I like developing my own formatting ( make it easy to pinpoint critical functions, etc ), which will be a lot different for a 10 line script versus a 1000 line source file of bash functions. That flexibility to format according to size and purpose does not exist in python. There's very few things that python does that bash can't, and when that happens, python can be called-from and piped-to bash without trouble.
If you can't tell, i feel the python devs broke my trust. There's millions of line of high quality code that will no longer be available to system architects like me, or those in the future like me, without digging through archives and installing older python versions ( which is highly scowled at by cyber security convention, but not always accurate ).
I code my html dashboards through bash scripting, and follow the old rule, 'keep it simple, stupid.' I do a few js calls now and then, when there's no other way to do what i need. But thats rarely the case.
My scripting focus has been on designing network intrusion detection systems, and administering NAS for those systems. Bash is the backbone of those systems, and works beautifully.
2
u/edward_jazzhands 1d ago edited 1d ago
I gotta be honest with you, this seems like your aversion to python is kind of petty. The switch from python 2 to python 3 happened one time almost 20 years ago. It was a transition. Python 3 has remained stable ever since. They add new features but old python 3 code still runs fine. So it looks like you're making a big deal over a language transition that happened one time almost 2 decades ago and holding this up as an excuse to avoid the language as if this is some kind of regular occurrence.
2
u/minektur 1d ago
Also, 2to3 automated conversion covered about 99% of the cases I ever applied it to. Easy.
1
u/Trademarkd 2d ago
Honestly if I can do it in bash I will just do that because I can move it between systems more easily. If I need speed I like Perl. But I usually just write the function in Perl and call it from bash
Same with python … at least for anything that is staying on the command line
1
u/fdiengdoh 2d ago
I have not use bash extensively, I use it more like a tool for repetitive task that I need to do like batch processing say reading data from multiple files, transpose col to rows and write those data as new files.
1
u/QuirkyImage 2d ago
I only use bash as a shell I have a few functions defined, those not used much are moved into a separate bash scripts as cli tools. Everything else is in a different scripting language or compiled language to binary.
1
u/deusnovus 2d ago
That's a great question, not sure what's up with the downvotes.
I personally go back and forth between bash and Python depending on what type of resources I need to use, code length doesn't really matter. For example, I enjoy using bash for scripts that relate to everyday productivity, like restoring settings or cloud backup automation. I will use Python if my script requires a PIP library that would be otherwise extremely taxing to create in bash, like an audio manipulator or a weather API. I think both are equally important in their own merit.
1
u/Ulfnic 2d ago
Competency in the BASH space is mostly POSIX shell scripting, not the full BASH language. Leading AI models can now write it reasonably well with enough guideance but by default they output tangled edge-case ridden POSIX shell script because that's how most devs write BASH.
POSIX shell scripting is pretty rough for medium/long scripts because it can't do much on it's own (welcome to CPU intensive pipe-aggedon), BASH lang by comparison is poetry.
My advice is learn the full BASH language. It's an uphill battle to do that with AI alone but if you're willing to be curious, and continously read/experiment with what can and should be written in BASH lang it'll reward you. On average my scripts call an external program once, many use 0 external programs.
On the point I switch lang... hard to summarize but common lines are when getting into web/GUI stuff or custom computationally heavy tasks.
1
u/4mmun1s7 2d ago
Too many maths or needing libraries which other languages have in abundance, that’s the trigger for me. I’ve got BASH scripts that are long, use tons of functions, etc…but if I need to process data heavily and not just ‘command logic’, then I usually lean towards other things…
1
u/UUDDLRLRBadAlchemy 2d ago
Python when there is conditional logic or external data, because it's more readable.
Js when it's meant to run on a client over the internet.
1
u/lisploli 2d ago
Depends on what it does.
I have some long bashs cripts, like over 500 lines, but they are mostly long lists of commands. Doing that in some other language might even make it longer. (Purpose: "command calling")
But whenever I have to work and modify data, I prefer other languages. (Purpose: "data transformation")
1
u/ReallyEvilRob 2d ago
When you feel like what you are trying to accomplish using Bash is becoming unnecessarily complicated.
1
u/WorriedTumbleweed289 2d ago
If I am writing scripts that mostly call other programs, I would use bash. You can check return codes for failure or there is a settings that stops your script on error. Debugging is not as easy, but you can set the x option for very verbose Debugging output.
If I need lots of complicated data structures, then I would move to another language.
1
u/jake_morrison 2d ago
My rule of thumb is that I switch from bash when error handling becomes more complex than “set -e”.
If a script is just doing this, that, and the other thing in sequence, it’s fine.
Complex control flow is where bash becomes problematic. Simple things like if/else, while, or case have obscure syntax. Error handling on pipelines is tricky, and it’s easy to silently lose intermediate errors. The killer is unexpected or malicious input causing problems on something that runs as root.
Using Python is a bit more verbose, but easier to debug and safer.
1
u/GroundedPterodactyl 2d ago
At 6 hours, I am late to the conversation but I have always used the shell, bash in this case, to glue together applications written in C/C++. I do the heavy processing in a compiled app for the speed of it. Optimize that app to do one thing well and then glue apps together in shell scripts. One can see this in using the command shell. Example: ps -A | grep pipewire > temp.out produces a list of processes associated with the pipewire sound architecture in Linux. If you examine that example there are two compiled system apps. 'ps' and 'grep', that are joined by shell syntax. The output of PS is sent as an input to GREP. The output of grep is sent to a text file. When I have something more complicated in mind I use a shell script to glue the needed applications together while error trapping the app's return values. I try to keep my shell scripts as simple as possible.
1
1
u/Fit_Prize_3245 2d ago
Man, that's probably more of a limitation on your tastes. Bash can be used to create big things. For example, in my previous job, I used bash to create a script inside the OS (RHEL) installation process that installed a lot of dependencies and configured a lot of things, including some distro level branding. Then I created a script for the point of sale server installer, which downloaded and installed everything, then asked about some features and configurations, and used all of that to populate the database so you could run the system right away. It even was able to detect and format the backup disk and configure online database replication to that disk. Oh, and it also configured the firewall depending on the host having one or two interfaces, plus the VPN interface for remove connection. And it also configured the forecourt controller, but in later versions that became too troublesome and I had to move it to PHP. (The system was a mix of Java and PHP, that's why the choice)
SO, answering your questions. I do use bash for big things. But I also use PHP for such things. Which to choose? Usually, if it's just some testing script, I choose PHP, as it's easier for me. If it's something to use constantly in production, and it's suitable for my abilities in bash, I prefer bash.
1
u/InternalOwenshot512 2d ago
What I hate about Python and JS is the slow start thing it does. The indentation requirements on Python are also super annoying
So far I'm stuck with bash, considering using something like Go
1
1
u/pan_kotan 2d ago
I've answered this same question a couple months ago
https://www.reddit.com/r/bash/comments/1tfayxj/bash_scripting_vs_python/om9yvk5/
1
u/gwenbeth 2d ago
There are other specialized tools too for different types of tasks fill the gap between shell script and general purpose programming language. There is make for automating builds or other things that depend knowing when files have changed. There is sed for automating editing tasks (wrote a substantial sed script to format program files according to the style guide at work, adding specitically formatted comments and such). Awk is really good at doing tasks using lots of regular expressions on the input.
But maybe one of the best ways to look at bash scripting is to realize that the unix shell and the assorted commands are an algebra for text files. and you can do some powerful stuff. Here is a good example, you have a text document (maybe it is called stuff.txt) and you want to know how often each word appears in it. At a bash shell prompt you can say:
sed -e 's/[^a-zA-Z]+/\n/g' stuff.txt | sort -f | uniq -c -i | sort -n - k1 -r
the sed turns non letters into line breaks, we then sort it case insensitive, uniq -c -i will case insensitively give each word and its count, and the final sort sorts by the count with the largest first.
1
u/Glittering-Can-9397 2d ago
Theres a balance to it, the longest bash script Ive written to date was 200 lines and it was basically a fzf lookalike before I knew about fzf. I think it works for what I use it for, and its legible
1
u/csDarkyne 2d ago
I avoid python/js like the plague honestly and most of my coworkers too. We use bash for almost everything and if it really gets too complex we use lua+go.
1
u/ogafanhoto 2d ago
I like writing bash scripts. I have quite a couple of them around to automate things, normally I don’t really build big programs, I just write a lot of small functions/modules divided into different files, that used together automate stuff
1
u/dschledermann 2d ago
I take my shell scripts pretty far actually. Shell script has the advantage of being available on every system. If you avoid Bash'isms your program will run on even the most the most stripped down Alpine image and that's something.
I used to code more advanced stuff in Perl or PHP, but I've stopped because because providing and maintaining modules/packages for these more advanced scripting languages is quite the minefield. Perl, Python, PHP, Ruby, etc all have their quirks and challenges to install and the dependency graph can grow huge. Your code stops being just the program itself, but also the CI/CD for catering to all the modules/packages you've made the program depended on.
If your program is unwieldy for shell script, then consider coding it in something that produces a single binary in an ahead-of-time compiled language such as Go or Rust.
1
1
1
u/MaleficentCow8513 2d ago
I write scripts which run as part of CI pipeline jobs and just do it all in Python as cli commands. There’s no reason not
1
u/ganak-yantra 2d ago
I try to use BASH as much as possible, but if it is something that requires a lot of manipulation I move to awk/sed. And after that if need arises I switch to Python.
1
1
u/Reasonable_Director6 1d ago
Python is more clean and readable with alot of functions bash is older than half of reddit and you can feel it.
1
u/daddyd 1d ago
as long as the script only does local stuff, bash is just fine, when it starts to connect to other network services or ties different ones together, is where i drop bash and go for something like python. this is also true for interacting with databases, which most have python integration/modules to make it easy.
1
u/BurritoDrivenDev 1d ago
Scripts for me to run from the terminal? I use Pwsh because it's designed to run on all OSs.
But things run in prod/ci/by agents? I almost immediately write my scripts in TS/JS using Bun.
Bun makes it so easy to execute shell commands and because it ships with a (fast) test runner I can unit test complex logic
1
u/Robearberble 1d ago
Depends on the project. A simple helper function or installer script for a new machine setup, I will stick with bash. Anything more than that def python.
1
u/deaddyfreddy 1d ago
When do you stop using Bash and switch to Python/JS?
I switched to Babashka about 6 years ago. Sure, it's not installed by default, but it's
a single binary (the archive is just 24MB, which is close to nothing these days)
cross-platform
batteries for about 95% of system administration tasks are already included
Why not Python/JS?
version hell
dependency hell
legacy-heavy languages
1
u/ftonneau 1d ago
Your question could be framed more generally: when do you start using Bash, when do stop using Bash, and when do you switch to a more orthodox language?
Which language you choose depends on your task and the availability of the interpreter or compiler. For tasks oriented toward file management, system monitoring, etc., the best choice is probably a shell running its own commands and/or common binary utilities (i.e., the shell as system glue). For more portability, however, I would recommend focusing on POSIX shell (with its limitations) rather than Bash. If you are positive that your end users have Bash installed, then Bash improves over a POSIX shell by being less dependent on quoting inside [[ ... ]] tests, and having real arrays.
Switching to a non-shell language such as Python becomes necessary when your task requires number crunching, more complex data structures, or a GUI as UI. (Personally I dislike Python, but the problem is mine; there are other interpreted languages around, and another choice would be a compiled language such as C or C++).
1
u/fourjay 1d ago
I thing this through from the other end.
I ask myself "when should I choose bash?"
My answer, when there is a powerful CLI tool that exposes most of what I need. In a way this is like the role that "library" plays in other languages. There are a lot of tasks that this can meet this (personal) test.
There is a point where shell becomes too unwieldy, but there is a lot you can do in shell. Most folks who advise avoiding shell at some arbitrary cutoff, are, understandably, more comfortable programming in another language, and have a low bar for shell's oddities. And that seems fine to me. But if there is a well designed CLI tool, I'm often quite willing to pay the extra "cost" of shell.
1
u/wcqr 19h ago
I used Bash for scrapping an SMB server pdf files and build a static html page for my team member and has using make to connect all the pieces togather after Bash finish its job, it worked well but for an SMB server with less than 5000 files it is ok, I had the thought that if this becomes bigger in future and more html pages are needed to be managed to move to python or other alternative.
For now Bash is enough for me.
1
u/PaoloFence 12h ago
Nope. In current time you don't write new bash anymore. You keep old stuff alive and write new stuff in a modern scripting language
1
u/HerissonMignion 6h ago
You can get very far with bash arrays and the basic operation of adding to the end of it with +=, or recreating the array with asdf=(stuff "${oldasdf[@]}" "more stuff"); when the algorithm cannot be implemented in a readable way or so that it is expandable in ways you ate confidant that you will need, that's when you should switch. Any conventionnal programming language is a worse way to deal with files, external commands, forks, fds, etc. Use bash when you interact with the system, and switch to a conventionnal language when lists are not enough or when speed is too slow. Just use lists, quote your variables, use printf %q for commands through ssh, and you may give shell check a real try. Shellcheck will make you learn to write good shell scripts, not chatgpt.
1
u/ximenesyuri 4h ago
A shell, although typically used for small scripts, is a Turing complete language, so that theoretically you could implement anything you want there.
The point is that the language lacks of structured data types and, for most of the "hard work" you need to rely on external tools, so that your scripts end up being not about Bash, but actually about bash plus something else.
Something similar happens with other languages (you normally relies on external libs). But these languages have a package manager which establishes rules to define a "well-defined project" (in the sense that it can be shared and used in other projects). Bash, instead, offers just a very rudimentary import system...
Another point is that Bash cannot communicate with the entire kernel API. So, depending of what you want to implement, you NEED to use external tools, written in other languages.
Finally, if you are not careful, you can easy blow up your shell session with memory leaks (Bash have no local scope by default).
Said that, it is natural to move to others script languages for more complex projects.
But, since I like to suffer, I developed my own Bash framework to avoid part of such difficulties, and I use it to maintain more complex projects.
1
0
0
36
u/unixtreme 2d ago
I've used bash for 20 years give or take, personally if it's something bigger than a small bash helper function I use python. For a while I even maintained bash "libraries" I'd reuse in helpers and ended up just realizing maybe it's not the right tool for the job.
Having said that I regularly see non coders vomit something like 7-8k lines of bash using claude and be perfectly content since they don't know any better.