I wanted to like modules

We use a program called Environment Modules to manage third-party software on Shaheen. It’s basically softenv, if you’re more familiar with that, but wasn’t written by MCS.

Anyway…

I’ve been working on documentation and overall transparency to our users recently, and I found myself wanting to generate documentation from our modules for publication on our wiki.

It’s a fairly typical shell script, and nothing unexpected.

#!/bin/bash

source /etc/profile.d/modules.sh

function main
{
    dest=$1
    mkdir -p $1
    modules=`module avail -t 2>&1 | grep -v :$`
    (
        echo "> automatically generated at" `date`
        echo
        echo "# Modules available"
        echo
        for module in $modules
        do
            echo >/dev/stderr $module
            echo "* \[[$module|$module]]"
            mkdir -p ${dest}/`dirname $module`
            mdwn=${dest}/${module}.mdwn
            module_to_mdwn $module >$mdwn 2>&1
        done
    ) > ${dest}.mdwn
}

function module_to_mdwn
{
    module whatis $1
    echo
    echo "# Module help"
    module help $1
}

main $@

The first annoyance comes from the fact that most output generated by modules is put on stderr. I’d argue philosophically that the right place for this output is stdout, but it’s not too big of a deal: just a few 2>&1 redirects.

Or so I thought. Output (other than the logging I’m doing intentionally) appears at the terminal. Output that I intended to redirect to the markdown file.

I spend an hour or so trying various placements of parenthesis, expecting that this has something to do with shell functions (module_to_mdwn, or even the module command itself). Debugging further leads me to the end of the road:

/usr/bin/modulecmd bash help $module &>/dev/null

module is a shell function that calls modulecmd under the covers. That command produces seemingly impossible output at the terminal.

Time for strace:

[...]
write(2, "\t*** No Module Specific Help for"..., 43 *** No Module Specific Help for INTEL ***) = 43

It’s writing to fd 2, which should be stderr redirected to /dev/null. What’s happening to those file descriptors?