Bash Concurrent; Bash Error Handling; The Oxford Principle

Being a macOS user, Zsh is the daily terminal driver, and I do like many of the quality-of-life improvements it offers. However, Bash is the universal shell script baseline, and I have never quite been able to put my finger on “why”, but it’s just so much fun to “program” in.

So, today, we cover an ingenious function that runs tasks in parallel and displays pretty output as they complete, a methodology to make bash scripts less brittle and more readable by doing “proper” error handling, and a command prefix principle that makes it brilliantly straightforward to keep track of and use your homegrown utilities.

No TL;DR since me, the human, just kind of did that ^^.

---

Bash Concurrent

One unfortunate aspect of my job is that I’m often waiting on queries from Elasticsearch to finish so the resultant horrible JSON can be processed. I end up having to use the scroll capability in ES so download many (sometimes dozens or hundreds) individual JSON files that need to then be post-prcessed into a sane, rectangular structure.

I used to wait until they were all done to do said post-processing since I didn’t feel like writing idiomatic and comprehensive scripts around parallel, especially since one has to fully take into consideration the methodology in the middle section of this tome when doing so, and I choose to not have time or patience to deal with that when it comes to working with ES.

However, concurrent has definitely changed things up a bit. It’s a Bash function that runs shell tasks in parallel while giving you a clean, real-time status display showing which tasks are running, completed, or failed. So, it’s both a declarative and lightweight alternative to GNU parallel or make -j (💙) but with a built-in TUI that shows progress.

I used “declarative” in that paragraph since concurrent manages dependencies between tasks, provided you tell it to do so. You can declare that task B should only start after task A succeeds, chain groups of tasks with --and-then, or run everything sequentially. It handles the orchestration and gives you visual feedback without needing to wire up background jobs, traps, and wait logic yourself.

There are mamy other use cases besides my data processing one. This is a perfect function to use when provisioning workflows where you need to create infrastructure in a specific order, build pipelines with parallel compilation steps, or any situation where you’re running a bunch of shell commands with interdependencies and want to see what’s happening without tailing a dozen log files.

It logs each task’s output to timestamped files, supports a compact mode for large task counts, and degrades gracefully to non-interactive output when there’s no tty. Oh, and it has a forking limit (default 50) that keeps you from accidentally fork-bombing yourself.

Completely brilliant.

---

Bash Error Handling

I am super lazy when it comes to building bulletproof Bash scripts. I will 100% make time to do so for truly mission-critical scripts, but I’m usually bolting on kludged error/exception handling idioms to scripts that turn out to not be one-offs.

Mathis Van Eetvelde lays out a great argument that we/I should definitely care about error handling in Bash, and provides solid guidance for de-brittle-izing our precious .sh files.

The post walks through exit codes, the various ways to catch failures (if/else, || operator, trap), and the common pitfalls that bite people in production, such as pipes silently swallowing errors, unset variables not triggering failures, and the default behavior of bash just plowing ahead after a non-zero exit.

The practical takeaway is the venerable set -Eeuo pipefail line at the top of your scripts, which enables exit-on-error, unset variable detection, pipe failure propagation, and ERR trap inheritance into functions/subshells. The post does a decent job explaining “why” each flag matters rather than just telling you to cargo-cult the incantation.

Aside: Ben Zanin made a great point on Mastodon that we should all use the less human-hostile set -o errtrace -o errexit -o nounset -o pipefail (I made a slight edit to accommodate Mathis’s set idiom). Someone encountering this for the first time can man bash and search for “errexit” and immediately find the relevant section. Searching for “-e” in the bash manual is an exercise in frustration given how many unrelated hits you’ll get.

The last section pivots into alerting, which is where the author’s product pitch lives (is is a freemium startup vendor blog, after all). The general advice is sound though: scripts running on cron or in unmonitored namespaces need to fail loudly, not silently.

It’s worth noting that the set -e footgun they mention (where capturing $? after a failing command causes early exit before you can inspect the code) is a real and frequently encountered problem. That interaction between -e and exit code capture is one of the reasons some people avoid set -e entirely and prefer explicit error handling throughout.

---

The Oxford Principle

I 💙 it when posts from nearly two decades ago survive link rot, and show that there’s still plenty to learn from historical knowledge drops.

In this 2009 post, Brandon Rhodes advocates for prefixing all personal ~/bin/ scripts with a comma character. The comma is one of the few unshifted keyboard characters that has no special meaning to the shell, can’t collide with system command names, and works perfectly with tab-completion, so typing ,<tab> instantly lists all your custom commands.

Simple idea with zero overhead that’s held up for over fifteen years now.

Brandon’s post is a quick read, and while you’re there, check out the rest of his stuff. NOTE: If you do take said advice, I will not apologize for the rabbit holes I just sent y’all down.

---

FIN

Remember, you can follow and interact with the full text of The Daily Drop’s free posts on:

• 🐘 Mastodon via @dailydrop.hrbrmstr.dev@dailydrop.hrbrmstr.dev
• 🦋 Bluesky via https://bsky.app/profile/dailydrop.hrbrmstr.dev.web.brid.gy

☮️