Photo by jesse orrico / Unsplash

Unix Tools for Testers

Tools Jan 6, 2025 (Jun 21, 2025) Loading...

A series on how I use these as a tester – probably to be used in Workroom PlayTime, and to support workshops on toolsmithery.

Published info and exercises

It's good to know how these simple tools can be used together.

Example 1

  • It may be that you've got a log from a test system and you want to extract all lines that refer to a particular transaction that seemed problematic. You'd grep.
  • Maybe you've got several logs, and you need to look for every reference. You could merge them while paying attention to the timestamps with a sort, then grep, and you'd join those two with a pipe |.
  • Maybe those logs are remote, but you can get them with cURL. You | them into sort. Maybe you want to add a suffix to each line, so that you can see the source. You curl | sed | sort | grep
  • Maybe you can't read the pages of scrolling weirdness. You | the grep into tail, to see the last 10 lines.
  • At the end, you've got a single hard-to-read but repeatable command curl | sed | sort | grep | tail – you use it once and throw it away. Or keep it in a handy spot.

Example 2

You've got a huge json record, containing some slice of a data state of a whole system – you want summaries for some test accounts, but it's got sooo much more. The json's invalid: it's got bad escapes in several places, but always in the same way and not anywhere you're interested in. You use cat to send the contents to sed and knock out the offending bits with a regular expression, send the now-valid json to jq and pick out the repeating parts you need. You want parts of all of them in a table – you tell jq to pick out what you need, transform the dates to human-readable, and output as a csv . Because this is a command, you make it part of the start and end of every big test run and drop the manageable files into... I dunno, Excel? so that you can see the context of each account as you explore.

Lots of utilities do text processing of some sort – filtering, sorting, aggregating. To get that text, other utilities pull information from files, the file system or processes. Information is passed from one utility to another with a handful of operators, and the end result is typically dumped into an editor or a file for someone to play with – or for another process to pick up.

As a tester, I've used these tools to check changes to my environment, to pick out specific data, to gather context for my exploration, to watch what's going on as I work, to filter the firehose.

Testers don't (typically) write stuff for permanent deployment: their tools are ephemeral. Those tools can still be shared, reused and tuned. However, capabilities built with these general tools are often so specific – and so hard to interpret – that they don't get shared, and get remade rather than reused.

By knowing what's possible, we can build things that are practical, and I hope that we will share and learn from each other.

It's a great time for testers to get better at using tiny tools: The rise of StackOverflow and blogging has led directly to a greater exchange of tricks, and LLMs, trained on these freely-available sources, are extraordinary at helping me to swiftly get to the tooling I need. I can ask, iterate and learn about what can be done, and options around how to do it, without getting too stuck on syntax and cluelessness.

Some sources:

Member reactions

Reactions are loading...

Sign in to leave reactions on posts

Tags

Comments

Sign in or become a Workroom Productions member to read and leave comments.

James Lyndsay

Along with Bart Knaack

Getting better at software testing. Singing in Bulgarian. Staying in. Going out. Listening. Talking. Writing. Making.

Great! You've successfully subscribed.
Great! Next, complete checkout for full access.
Welcome back! You've successfully signed in.
Success! Your account is fully activated, you now have access to all content.