We use a program called Environment Modules to manage third-party software on
Shaheen. It’s basically softenv, if you’re more familiar with that,
but wasn’t written by MCS.
Anyway…
I’ve been working on documentation and overall transparency to our
users recently, and I found myself wanting to generate documentation
from our modules for publication on our wiki.
It’s a fairly typical shell script, and nothing unexpected.
#!/bin/bash
source /etc/profile.d/modules.sh
function main
{
dest=$1
mkdir -p $1
modules=`module avail -t 2>&1 | grep -v :$`
(
echo "> automatically generated at" `date`
echo
echo "# Modules available"
echo
for module in $modules
do
echo >/dev/stderr $module
echo "* \[[$module|$module]]"
mkdir -p ${dest}/`dirname $module`
mdwn=${dest}/${module}.mdwn
module_to_mdwn $module >$mdwn 2>&1
done
) > ${dest}.mdwn
}
function module_to_mdwn
{
module whatis $1
echo
echo "# Module help"
module help $1
}
main $@
The first annoyance comes from the fact that most output generated by
modules is put on stderr. I’d argue philosophically that the right
place for this output is stdout, but it’s not too big of a deal: just
a few 2>&1 redirects.
Or so I thought. Output (other than the logging I’m doing
intentionally) appears at the terminal. Output that I intended to
redirect to the markdown file.
I spend an hour or so trying various placements of parenthesis,
expecting that this has something to do with shell functions
(module_to_mdwn, or even the module command itself). Debugging further
leads me to the end of the road:
/usr/bin/modulecmd bash help $module &>/dev/null
module is a shell function that calls modulecmd under the
covers. That command produces seemingly impossible output at the
terminal.
Time for strace:
[...]
write(2, "\t*** No Module Specific Help for"..., 43 *** No Module Specific Help for INTEL ***) = 43
It’s writing to fd 2, which should be stderr redirected to /dev/null.
What’s happening to those file descriptors?