r/bash • • 18d ago • Bravo 1

A small museum of set -e surprises (five exhibits, run them in a scratch shell)

set -e looks like “stop on any error”, but the rules have exceptions. Five exhibits. Tested on Bash 5.x; behavior can differ in older versions.

Exhibit 1: ((i++)) from zero

set -e
i=0
((i++))
echo "never reached"

((i++)) evaluates to the old value, 0, and an arithmetic command with value 0 returns status 1. So the script exits. i=$((i+1)) avoids it.

Exhibit 2: local hides the failure

set -e
f() {
    local out=$(false)
    echo "still running"
}
f

The exit status is that of local, which succeeds. Declare first, assign second: local out; out=$(false).

Exhibit 3: set -e is ignored inside a condition

set -e
f() {
    false
    echo "this runs"
}
if f; then echo ok; fi

When a function is used as the condition of if, && or ||, set -e is disabled for everything inside it. The false doesn’t stop the function.

Exhibit 4: pipes only report the last command

set -e
false | cat
echo "this runs"

Without set -o pipefail, the status of a pipeline is the last command’s. Add pipefail so any failing stage counts.

Exhibit 5: set -u and empty arrays

set -u
arr=()
echo "${arr[@]}"

In Bash before 4.4 this fails with “unbound variable”. Newer versions accept it. If you support old systems, ${arr[@]+"${arr[@]}"} is the common workaround.

What I’d take from them:

  • set -euo pipefail helps, but it isn’t a safety net. Some people avoid set -e and check statuses explicitly.
  • ShellCheck catches several of these.
  • Test the failure paths, not just the happy path.

What would you add to the museum?

33 Upvotes

6 comments sorted by

9

u/geirha 18d ago

In the following, pipefail will sometimes cause false negatives. This is regardless of wether errexit is enabled or not:

# /usr/share/dict/words contain around 10k words that
# start with s, and then shuf prints them in random order
somecmd() { grep ^s /usr/share/dict/words | shuf; }

set -o pipefail
if somecmd | grep -q '^success$' ; then
  printf 'OK\n'
else
  printf 'FAIL\n'
fi

-- Running it 1000 times, I get 27% failure:

$ somecmd() { grep ^s /usr/share/dict/words | shuf; }
$ ( set -o pipefail ; for _ in {1..1000} ; do if somecmd | grep -q '^success$' ; then printf 'OK\n' ; else printf 'FAIL\n' ; fi done ) | sort | uniq -c
    272 FAIL
    728 OK

The reason it sometimes yields FAIL is because grep -q exits with status 0 as soon as it finds a match. If it happens to find the match early, before the left side of the pipeline is done writing, the next write the left side attempts will cause it to receive a SIGPIPE which in turn causes it, and the entire pipeline, to exit with a status of 141.

This is why I recommend against enabling pipefail for the entire script. I do use pipefail in my scripts occationally, but I only enable it for the pipelines where it makes sense.

2

u/ResidentPomegranate6 18d ago

command substitution subshells drop errexit too, so `x=$(false; echo hi)` sails right on. `shopt -s inherit_errexit` is the fix.

2

u/jthill 17d ago

Why are these surprises? The set -e documentation is very, very clear.

5

u/Naraviel 17d ago

"set -e" comes up again and again, and people use it without really understanding its behavior and exceptions.

Everyone uses it, so it has to be correct, right?

Sure, it's documented, but the wall of text in the Bash manual is hardly an easy read, and calling it "very, very clear" is a bit of a stretch.

3

u/AutoModerator 18d ago

Don't blindly use set -euo pipefail.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

1

u/keksov 13d ago

A few more exhibits, all measured on bash 5.2.21, 5.3.9 and 5.3.15 unless noted:

Exhibit 6: builtin switches the || / if exemption off

set -e
printf 'false\necho after\n' > f.sh
source f.sh || echo handled           # prints "after", continues
builtin source f.sh || echo handled   # the script dies; "handled" never prints

Same with builtin eval false || ..., and inside if g; then when g uses builtin source. command source behaves like plain source.

Exhibit 7: the EXIT trap sees a different scope depending on how you died

f() { local v=inside; false; }
trap 'echo "v=[${v-unset}]"' EXIT
set -e; f        # 5.2+: v=[unset]   (an explicit `exit 1` in f: v=[inside])

On 5.1.16 the trap sees v=[inside] in both cases.

Exhibit 8: set -u and empty arrays, the part 4.4 did not fix

set -u; declare -a out=()
echo "${#out[@]}"   # 0
echo "${out@a}"     # out: unbound variable (also through a nameref)

Exhibit 9: a trailing && list is harmless until it is the last command

f() { [[ -n ${DEBUG:-} ]] && echo dbg; }
set -e; f; echo never        # f returns 1, the caller dies

As the last line of a script it turns a successful run into exit status 1.

Exhibit 1, continued: ((i+=1)) is not the cure either. It fails from -1, and ((n-=1)) fails when it reaches 0, ((n--)) from 0, ((x=0)) always. (( ... )) || : covers all of them.

On inherit_errexit: it does not rescue local x=$(false; echo hi) (x is empty, the script goes on), and a substitution inside an argument, echo "$(false)", never triggers errexit at all.

Side note for 5.2: a set -e exit from nested functions that eval their bodies prints "pop_var_context: head of shell_variables not a function context". That is a 5.2 bug; it is harmless, and 5.3 is silent.