Posts about technology (old posts, page 1)

MobileOrg for journaling

I suppose that now is as good a time as any to point out that I’m using mobileorg’s capture feature to take these notes. The asyncronous syncing of the system should make it relatively easy to port these notes back into a desktop when I return (albeit via my slice). The plaintext format leaves me feeling in control of the content, too.

There will be a bit of an impedance mismatch between org and markdown, but it shouldn’t be too bad. Who knows: I might even be able to automate the process.

Oracle shoves MySQL’s square peg into an enterprise-y round hold

Oracle’s has a new tiered pricing model for MySQL. First on the list: InnoDB requires moving from the free “Classic” edition to the $2000/year (for up to four sockets) “Standard” edition.

Now all Oracle has to do is find a way to charge people for the privilege of dropping MySQL in favor of Postgres.


Full, non-supported releases are available as the “Community” edition.

Did MySQL have a contribution copyright assignment policy? Will Oracle be able to create a proprietary fork of MySQL, or are they bound by the terms of the GPL?

MobileOrg push by way of git

Following up from yesterday, I now have a fully automated MobileOrg environment. Simply pushing into my org file git repository provokes a post-receive hook that calls a bit of elisp:

#!/usr/bin/emacs --script

(load "~/.emacs")
(setq org-mobile-directory "~/public_html/org")

Setting org-mobile-directory in the hook allows me to keep it more generalized in .emacs (that is, via /ssh:, so I can run org-mobile-push and, especially, org-mobile-pull from a desktop.

Now, when I do a git push, I also get this (among other, more verbose logging) in my stderr:

remote: Files for mobile viewer staged

MobileOrg push

I just spent the last hour and a half fighting with webdav, emacs, and MobileOrg. I thought that I had all of this settled a while ago, but, as before, org-mode just keeps surprising me with new functionality that I hadn’t yet realized that I needed. Now I can capture from my mobile, as well as update todo states.

Next up, I want to do the org-mobile-push from the emacs running on my slice, rather than from Aquamacs running on the desktop; and I want to dispatch it as a post-commit hook in my bare orgfile git repo.

second LISA submission

Last night I submitted my second paper to LISA. (The first was as coauthor for a paper that was not accepted). I had to hurry when I found out that the terms of submission had changed since last year, but at least I got it in (and by the third attempt, I had come to terms with my distilled “extended abstract”).

Now I have to do is wait to see if this paper is accepted. I should hear back by 1 July.

I wanted to like modules

We use a program called Environment Modules to manage third-party software on Shaheen. It’s basically softenv, if you’re more familiar with that, but wasn’t written by MCS.


I’ve been working on documentation and overall transparency to our users recently, and I found myself wanting to generate documentation from our modules for publication on our wiki.

It’s a fairly typical shell script, and nothing unexpected.


source /etc/profile.d/

function main
    mkdir -p $1
    modules=`module avail -t 2>&1 | grep -v :$`
        echo "> automatically generated at" `date`
        echo "# Modules available"
        for module in $modules
            echo >/dev/stderr $module
            echo "* \[[$module|$module]]"
            mkdir -p ${dest}/`dirname $module`
            module_to_mdwn $module >$mdwn 2>&1
    ) > ${dest}.mdwn

function module_to_mdwn
    module whatis $1
    echo "# Module help"
    module help $1

main $@

The first annoyance comes from the fact that most output generated by modules is put on stderr. I’d argue philosophically that the right place for this output is stdout, but it’s not too big of a deal: just a few 2>&1 redirects.

Or so I thought. Output (other than the logging I’m doing intentionally) appears at the terminal. Output that I intended to redirect to the markdown file.

I spend an hour or so trying various placements of parenthesis, expecting that this has something to do with shell functions (module_to_mdwn, or even the module command itself). Debugging further leads me to the end of the road:

/usr/bin/modulecmd bash help $module &>/dev/null

module is a shell function that calls modulecmd under the covers. That command produces seemingly impossible output at the terminal.

Time for strace:

write(2, "\t*** No Module Specific Help for"..., 43 *** No Module Specific Help for INTEL ***) = 43

It’s writing to fd 2, which should be stderr redirected to /dev/null. What’s happening to those file descriptors?

Safari bookmarks, git repositories, and automatic updating

I recently added ~/Library/Safari/Bookmarks.plist to my homedir git repository in the misguided hope that it would be a magically sensible way to keep my bookmarks sync’d between my various systems. This evening was the first real test:

Kay:~ janderson$ pgit pull
remote: Counting objects: 9, done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 5 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (5/5), done.
From ssh://
   9d1b3c1..5fdae45  master     -> origin/master
Updating 9d1b3c1..5fdae45
 Library/Safari/Bookmarks.plist |  Bin 1951 -> 2498 bytes
 1 files changed, 0 insertions(+), 0 deletions(-)
Already up-to-date.

Not only did the day’s changes propagate to my local system, but Safari implemented the change immediately, in the current running process, in the current window, without intervention. I’m sure this has something to do with fsevents, but it’s just one more example of the extent of behavior you can get away with if you implement a system properly in the first place. (After all: it’s not as though Apple had this use case in mind.)

Of course, the true test will be how well the file format copes with an n-way merge… but I’ll cross that bridge when git blows up in my face.


I use a couple of git repositories in my home directory to track and sync my configuration and org files across multiple platforms. Originally I used a single repository for everything, tracked at ~/.git, but when I split the two concerns I knew that I would want a simple way to manage all of my local git repositories at once. Inspired by my use of existing pssh-style commands, I wrote a quick (not quite so) parallel pgit:


for repo in ~ ~/org
    echo 1>&2 $repo
    cd $repo
    git $@
    cd - >/dev/null

I want to improve this at some point with a configurable repository list (likely a custom section in ~/.gitconfig) and actual parallel execution, but at least for day-to-day, this has already been satisfying to use.

clusterbank migrates to civilfritz

Way back when I worked at Argonne I developed a little program called clusterbank that we used to manage resource allocations. Now that I’m at KAUST, we need the same thing again, so I’ve drudged the code up from the depths of one of my hard drives. (It was actually pretty nasty, and I’m lucky to have found as recent a copy as I did: the first copy that I tried to use had corruption that made “git fsck” fail.)

Anyway, I’m going to have to do a bit of coding to make it work at KAUST (surprise! it didn’t just work out-of-the box) so I’ve migrated the most recent branch that I could find to civilfritz, its new canonical home.

I was told that any code developed on DOE time has to be open-sourced with a permissive license (e.g., BSD). I don’t know what I should have actually done to make that happen at ANL, but I’m sure no one will complain if I just stick the BSD license on the code and call it good for now.

description of Shaheen from the WatsonLinux decommissioning

I just sent out the word that WatsonLinux is being decommissioned, and managed a relatively good sales pitch for Shaheen in the process. For those of you who wonder what I do…

Shaheen is powered by a 16-rack (65536-core) IBM Blue Gene/P system and a 96-(soon 128-)node IBM System x cluster. It currently ranks at #18 on the TOP500 list [1] of the world’s most powerful supercomputers, and remains the most powerful supercomputer in the middle east, capable of 222.82 teraflops peak (190.90 sustained).

Built with the environment in mind, Shaheen ranks #9 on the TOP Green500 list [2], providing 378.77 megaflops per watt.

If you are interested in using Shaheen in your research, contact our support desk at