Language: Bash · Versions: GNU Bash 5.2 (set -euo pipefail)
The problem
I run scripts with set -euo pipefail. Two subtle pipeline behaviors cause production scripts to silently lose state or abort with exit code 141 (SIGPIPE):
1. In generate_lines | grep "WARN" | head -n 1, as soon as head -n 1 reads the first matching line and closes its stdin, generate_lines / grep receives SIGPIPE (exit status 141) on its next write. Because pipefail is enabled, the entire pipeline returns 141 and set -e immediately aborts the script!
2. In cmd | while read -r line; do count=$((count + 1)); done, the while loop runs in a subshell (unless shopt -s lastpipe is set in a non-interactive shell), so count is still 0 after the loop!
Minimal example
#!/usr/bin/env bash
set -euo pipefail
# Trap 1: SIGPIPE (141) under pipefail + set -e when producer writes > pipe buffer (64KB):
first_match=$(seq 1 100000 | head -n 1)
echo "First: $first_match"
# Trap 2: Subshell variable loss:
count=0
printf "a\nb\nc\n" | while IFS= read -r _; do
count=$((count + 1))
done
echo "Count: $count"
Expected
First: 1 and Count: 3 without aborting under set -euo pipefail.
What happens instead
seq 1 100000writes more than the 64 KB kernel pipe buffer beforehead -n 1exits, receivesSIGPIPE(128 + 13 = 141), and underset -euo pipefailaborts the script atfirst_match=...!- If Trap 1 is bypassed,
Count: 0is printed because the right-hand side of|runs in a child subshell.
What is the cleanest way in Bash 5.2 to keep set -euo pipefail enabled while avoiding SIGPIPE (141) false-failures on early-closing consumers like head -n 1 or grep -q?
Both premises hold. I'm reasoning from the Bash manual and POSIX pipe semantics, and I haven't run this.
Fixed script
#!/usr/bin/env bash
set -euo pipefail
shopt -s lastpipe # only takes effect when job control is off (the default in scripts)
# Trap 1: read from a process substitution, so no pipeline status exists
IFS= read -r first_match < <(seq 1 100000)
echo "First: $first_match"
# Trap 2: same idea, loop stays in the current shell
count=0
while IFS= read -r _; do
count=$((count + 1))
done < <(printf 'a\nb\nc\n')
echo "Count: $count"
Why Trap 1 happens
- When
headexits,seqgets SIGPIPE on its next write and exits 141. - With
pipefail, the pipeline's status is the rightmost non-zero one, so it is 141. - The assignment
x=$(...)takes the status of the command substitution, soset -eaborts. - One correction to your explanation: the problem isn't that
seqwrites more than 64 KB beforeheadexits. It is thatseqstill has output left to write afterheadhas gone, which is why it is a race. Output that fits in the pipe buffer won't trigger it reliably, so the script can pass in testing and fail in production.
Why the fix works
read < <(...)has no pipeline, sopipefailhas nothing to check.- Bash doesn't apply
set -eto a process substitution's exit status. readreturns 0 after one line, and the producer dies quietly on SIGPIPE.- The cost is that a genuine producer failure is also ignored. In Bash 4.4 and later you can
wait "$!"on the process-substitution PID if you need that status.
Other ways to keep a pipe
- Absorb only 141. Wrap the producer side so that 141, and nothing else, counts as success:
first=$( { seq 1 100000 | grep -E '5$' || [ "$?" -eq 141 ]; } | head -n 1 )
The inner pipeline's non-zero status is turned into 0 only if it is 141. A real failure (for example status 2) still propagates.
- Use a consumer that drains its input. sed -n 1p and awk 'NR==1' (GNU sed and awk both read to EOF) never close the pipe early. This is the simplest option for small or finite producers, but it is wasteful or hangs on infinite ones such as yes.
- For boolean tests. Avoid producer | grep -q under pipefail, because grep -q exits at the first match and has the same problem. Use grep pattern >/dev/null (reads everything), or grep -q pattern < <(producer).
- Avoid || true. It hides real failures from the producer.
Trap 2
lastpiperuns the last pipeline element in the current shell. It is ignored when job control is on, so it won't work interactively.done < <(...)works everywhere and doesn't depend on that.- A related trap:
((count++))returns status 1 whencountis 0, which aborts underset -e.count=$((count + 1))avoids that.