Posts about technology (old posts, page 5)

5 February 2015

Home

I finally got curtain brackets mounted in the family room, though no curtains are hung yet. Last night my drill battery died, and I ended up with a drill bit stuck in the wall; but now it’s out, anchors in, and brackets hung. Tonight, curtains.

Tech

My piratebox still won’t install the software from the usb stick, claiming that the storage device is full. Since there’s plenty of space left on the usb stick, I can only assume it’s talking about the internal storage. Maybe I flashed it incorrectly? Or maybe a previous failed install has left it without some space it exepcts to have? Or maybe I need to do some kind of reset?

In any case, I was able to telnet in, re-flashed the firmware manually, hit the reset button for good measure, and then tried installing again from install_piratebox.zip. This time it worked, and I have a “PirateBox” ssid once again following me around. I must admit, though: I’m a bit disappointed by the “new, responsive” layout. I think it’s really only the media browser that has improved.

I don’t know that I’m going to do anything about it today, but I’d really like to install OpenProject on civilfritz. I think it would help Andi and me work together to get projects done at home.

CU

We had a meeting with Peak and IBM storage about a possible replacement for the existing home and projects storage N-series system. They are proposing a black-box GPFS system “IBM StoreWise v7000 Unified” which I should look into further. They have also mentioned ESS nee GSS (hopefully still a grey-box solution), though they didn’t have many details on what that would look like. They weren’t even sure if it’s running Linux or AIX. (I’m hoping that their first instinct was wrong, and that it’s Linux on Power.)

We have a talk scheduled about upgrading the PetaLibrary. I specifically am of the opinion that we should be using the PetaLibrary to house home and projects. The way I see it, we should have two independent filesets, /pl/home/ and /pl/projects/, and just NFS export and snapshot each of them. We could even still have each home and project directory be a dependent fileset underneath, if we want.

I’m going to keep working on the user guide today. Hopefully I’ll have a batch queueing / slurm guide to give to Aaron by tomorrow.

I also need to work on a local passwords pam config to tick off the last of my performance plan post-its.

Before I do anything, though, I should look through org-mode and calendar to see what else I might be forgetting.

Understanding OpenStack networking with Neutron and Open vSwitch

I couldn’t figure out OpenStack’s networking system enough to get my instances’ floating IPs to work, even from the packstack --allinone host itself. I read the RDO document Networking in too much detail, but even that seemed to assume more knowledge about how things fit together than I had.

I eventually got some help from the #rdo irc channel; but I think the best documentation ended up being Visualizing OpenStack Networking Service Traffic in the Cloud from the OpenStack Operations Guide.

In the end, most of my problem was that I was trying to assign an IP address to my br-ex interface that conflicted with the the l3-agent that was already connected to the br-ex bridge. Literally any other address in the subnet that wasn’t also used by an instance gave me the behavior I was looking for: being able to ping the floating addresses from the host.

ip addr add 172.24.4.225/28 dev br-ex

Once that was done, I was able to configure NAT on the same host. This is described at the end of the “Networking in too much detail” document, and was echoed by the individual who helped me in #rdo; but I modified the POSTROUTING rule to identify the external network interface, p4p1. If the external interface is left unspecified, then even internal traffic from the host to the guests will be rewritten to the external address, which isn’t valid on the floating-IP subnet.

iptables -A FORWARD -d 172.24.4.224/28 -j ACCEPT
iptables -A FORWARD -s 172.24.4.224/28 -j ACCEPT
iptables -t nat -I POSTROUTING 1 -s 172.24.4.224/28 -o p4p1 -j MASQUERADE

Rebuilding civilfritz TLS

Some random notes on when I rebuilt TLS for civilfritz using gnutls and cacert.

  • https

  • ldap start_tls

http://www.gnutls.org/manual/html_node/certtool-Invocation.html

certtool --generate-privkey --outfile civilfritz.net.key

certtool --generate-request --load-privkey civilfritz.net.key --outfile civilfritz.net.csr

https://www.cacert.org

vi civilfritz.net.pem

certtool --certificate-info < civilfritz.net.pem

Subject Alternative Name (not critical):
   DNSname: civilfritz.net
   XMPP Address: civilfritz.net
   DNSname: www.civilfritz.net
   XMPP Address: www.civilfritz.net

$ cat civilfritz.net.pem /etc/ssl/certs/cacert.org.pem | certtool --verify-chain
Certificate[0]: CN=civilfritz.net
    Issued by: O=Root CA,OU=http://www.cacert.org,CN=CA Cert Signing Authority,EMAIL=support@cacert.org
    Verifying against certificate[1].
Error: Issuer's name: O=CAcert Inc.,OU=http://www.CAcert.org,CN=CAcert Class 3 Root
certtool: issuer name does not match the next certificate


$ cat civilfritz.net.pem cacert.org.pem | certtool --verify-chain
Certificate[0]: CN=civilfritz.net
    Issued by: O=Root CA,OU=http://www.cacert.org,CN=CA Cert Signing Authority,EMAIL=support@cacert.org
    Verifying against certificate[1].
    Verification output: Verified.

Certificate[1]: O=Root CA,OU=http://www.cacert.org,CN=CA Cert Signing Authority,EMAIL=support@cacert.org
    Issued by: O=Root CA,OU=http://www.cacert.org,CN=CA Cert Signing Authority,EMAIL=support@cacert.org
    Verification output: Verified.

Chain verification output: Verified.

Migrating from Apache to Nginx

I've had this hanging out as a draft for a while, and never got it polished up as well as I'd like; but in case it's useful to anyone, here's some notes on my recent migration to nginx.


I run civilfritz.net on a VPS, but I do my best to keep as low a monthly payment as possible. That means running on the smallest (lowest-memory) VM available from my provider: a 1GB Linode.

1GB used to seem like a lot of memory; but when I'm trying to run a Minecraft server alongside a preforking Apache server alongside a Salt master, it fills up quickly.

I've wanted to try moving to a lighter-weight webserver for a while; so today I'm porting my Apache config to Nginx.

sites-available/civilfritz.net

civilfritz.net runs as a pair of Apache virtual hosts to support http and https. I want the majority of the configuration between the vhosts to be identical, so I include a separate common configuration file in each.

The http vhost includes the common config, as well as a rewrite for the ikiwiki /auth section. (Authentication should only happen over https, but attempts to authenticate over http should be redirected there.)

# apache http vhost

<VirtualHost *:80>
    RewriteEngine on
    RewriteRule ^/auth(|/.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [R,L]

    Include sites-available/civilfritz.net-common
</VirtualHost>

The transition to nginx was pretty simple. The ikiwiki /auth section is a virtually equivalent rewrite rule, and the include directive is also similar.

# nginx http vhost

server
{
        listen 80;

        rewrite ^/auth(|/.*)$ https://$server_name:443$request_uri? permanent;

        include sites-available/civilfritz.net-common;
}

The https vhost also includes the common config, as well as the requisite ssl config. To support http basic authentication, an instance of pwauth is configured as an external authentication module, which proxies to PAM.

# apache https vhost

<VirtualHost *:443>
    SSLEngine on
    SSLCertificateFile /etc/ssl/certs/civilfritz.net.pem
    SSLCertificateKeyFile /etc/ssl/private/civilfritz.net.key

    AddExternalAuth pwauth /usr/sbin/pwauth
    SetExternalAuthMethod pwauth pipe

    <Location />
        AuthType Basic
        AuthBasicProvider external
        AuthExternal pwauth
        AuthName "civilfritz.net"
    </Location>

    Include sites-available/civilfritz.net-common

    <Location /auth>
        Require valid-user
    </Location>
</VirtualHost>

Again, the nginx vhost starts out similarly. Listen on tcp 443, initialize the requisite certificate and key, and include the common config.

pwauth is an Apache-specific interface, so I wasn't able to use it to proxy to pam in nginx; but the auth_pam module works well enough and, since I'm not trying to use PAM to auth directly against local unix files (I'm using sssd to access kerberos), I still don't have to run the server as root.

# nginx ssl vhost

server
{
        listen 443 ssl;

        ssl_certificate /etc/ssl/certs/civilfritz.net.pem;
        ssl_certificate_key /etc/ssl/private/civilfritz.net.key;

        include sites-available/civilfritz.net-common;

        location /auth
        {
                auth_pam "civilfritz.net";
                include fastcgi_params;
                fastcgi_pass unix:/var/run/fcgiwrap.socket;
                fastcgi_index ikiwiki.cgi;
                fastcgi_param REMOTE_USER $remote_user;
        }
}

The semantics of Nginx basic authentication differ from Apache. In Apache I was able to set AuthName globally (at /) and then require authentication arbitrarily at lower points in the tree. Here, the inclusion of the auth_pam directive implies an auth requirement; so I'll have to repeat the authentication realm ("civilfritz.net") anywhere I want to authenticate.

The biggest difference, though, is how Nginx handles cgi. Whereas Apache builds-in cgi execution for nominated files or directories, Nginx proxies all cgi execution through an external interface: here, fastcgi. A packed-in fastcgi_params file contains some useful default cgi environment variables, but omits REMOTE_USER. I set here so that ikiwiki can determine what user has authenticated.

sites-available/civilfritz.net-common

The vast majority of my local config is in the common file included by both vhosts.

# Apache initial config

ServerAdmin anderbubble@gmail.com
DirectoryIndex index.html
ServerName civilfritz.net
ServerAlias www.civilfritz.net

LogLevel warn
ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combined

DocumentRoot /srv/www/wiki

RewriteEngine on

Alias /robots.txt /srv/www/robots.txt

Alias /minecraft/overview /srv/www/minecraft-overviewer

<Location /users/janderson/private>
    Require user janderson
</Location>

<Directory />
    Options FollowSymLinks
    AllowOverride None
    Order deny,allow
    Deny from all
</Directory>

<Directory /srv/www>
    Order allow,deny
    Allow from all
</Directory>

<Directory /srv/www/wiki>
    AddHandler cgi-script .cgi
    Order allow,deny
    Allow from all
    Options +ExecCGI
    ErrorDocument 404 /ikiwiki.cgi
    ExpiresActive on
    ExpiresDefault "access plus 0 seconds"
    Header set Cache-Control "no-store, no-cache, must-revalidate, max-age=0"
    Header set Pragma "no-cache"
</Directory>

<Location /gitweb>
    Order allow,deny
    Allow from all
    DirectoryIndex index.cgi
</Location>

<Directory /home/*/public_html/>
    AllowOverride FileInfo AuthConfig Limit Indexes Options=ExecCGI
</Directory>

WSGIApplicationGroup %{GLOBAL}

New Nginx config

sites-available/civilfritz.net-common

index index.html;
server_name civilfritz.net www.civilfritz.net;

root /srv/www/wiki/;

location /
{
        error_page 404 /ikiwiki-404.cgi;
        expires -1;
}

location /robots.txt
{
        alias /srv/www/robots.txt;
}

location /minecraft/overview
{
        alias /srv/www/minecraft-overviewer;
}

location /ikiwiki.cgi
{
        include fastcgi_params;
        fastcgi_pass unix:/var/run/fcgiwrap.socket;
        fastcgi_index ikiwiki.cgi;
}

location /ikiwiki-404.cgi
{
        internal;
        include fastcgi_params;
        fastcgi_pass unix:/var/run/fcgiwrap.socket;
        fastcgi_param REDIRECT_URL $request_uri;
        # also needed to remove explicit 200
        fastcgi_param REDIRECT_STATUS 404;
}

location ~ /gitweb/(index|gitweb).cgi
{
        root /usr/share/;
        gzip off;
        include fastcgi_params;
        fastcgi_pass unix:/var/run/fcgiwrap.socket;
}

location /gitweb/
{
        root /usr/share/;
        gzip off;
        index index.cgi;
}

location ~ ^/~(.+?)(/.*)?$
{
        alias /home/$1/public_html$2;
        autoindex on;
}

Salt state

nginx.sls

nginx:

  pkg:
    - installed

  service:
    - running
    - enable: True
    - reload: True
    - watch:
      - pkg: nginx

civilfritz/www.sls

include:
  - nginx

[...]

/etc/nginx/sites-enabled/default:
  file:
    - absent
    - watch_in:
      - service: nginx

/etc/nginx/sites-enabled/civilfritz.net:
  file:
    - symlink
    - target: /etc/nginx/sites-available/civilfritz.net
    - require:
      - file: /etc/nginx/sites-available/civilfritz.net
    - watch_in:
      - service: nginx

/etc/nginx/sites-available/civilfritz.net:
  file:
    - managed
    - source: salt://civilfritz/nginx-sites/civilfritz.net
    - user: root
    - group: root
    - mode: 0644
    - require:
      - file: /etc/nginx/sites-available/civilfritz.net-common
    - watch_in:
      - service: nginx

/etc/nginx/sites-available/civilfritz.net-common:
  file:
    - managed
    - source: salt://civilfritz/nginx-sites/civilfritz.net-common
    - user: root
    - group: root
    - mode: 0644
    - watch_in:
      - service: nginx

/srv/www/wiki/ikiwiki-404.cgi:
  file:
    - symlink
    - target: /srv/www/wiki/ikiwiki.cgi

Raspberry Pi and Arduino | Christmas 2013

I spent a fair amount of time this Christmas hacking around on the Raspberry Pi with my electrical-engineer brother-in-law. On top of that, he gave me an Arduino Uno Starter Pack for Christmas. So now we have a great foundation of common ground between our neighboring disciplines.

His original project goal for the Raspberry Pi was, "A webpage I can use from my phone to click a button and make lights turn on and off." With the help of Pylons Pyramid and an existing GPIO Python module I was able to make a rudimentary GPIO web interface. It allows you to designate pins as input or output, and read or set the state of each pin. You can even (thanks to Twitter's Bootstrap) visit it from your phone, and it won't look all that terrible.

The Arduino is my first real foray into embedded programming (unless you count the rudimentary CPU I built in my undergraduate Computer Organization and Design course). At the suggestion of my brother-in-law and the Internet I set out to create the "Hello, world!" of electronics: a blinking LED. That said, the Arduino IDE on Debian was so easy to use (it's literally an apt-get install arduino away) that I wasn't terribly satisfied with the relatively minimal work it took to achieve my blinking light. It had been a while since I'd written much C, so I took the opportunity to make a more full-featured "Hello, world!" with the addition of Morse code, serial console logging, and analog encoding control.

When he returns from a visit with his own respective in-laws, we're going to brush up on my soldering skills by building the Adafruit proto shield. Here's hoping that projects like these keep us collaborating, despite the physical distance between us after the holidays are over.

arduino-hello-world.thumbnail.jpg

Documentation as a habit

People seemed to like this comment that I made at LISA 2013:

It’s important to see documentation as a habit, rather than as a task to be completed. Don’t ask how you can catch up on the backlog of documentation that isn’t written; just document what you do and answers to questions you’ve had to ask from here on out.

org-mode iPhone Agenda

I generally keep on top of my to-do list at work, thanks to my “keep everything in org-mode habit.” Since I spend all day at work in front of a computer, org-mode is always a few keys away, and it’s easy to review my agenda throughout the day and keep my lists up-to-date.

There was a time when that was true of my personal lists as well; but, for whatever reason, I don’t tend to have org-mode open at home any more. There will be times (like this instant) where I remember, and will open emacs, and will review my agendas; but, most often, that just leads to me continuing to procrastinate, given how far behind I already am.

I’m pretty sure that what I need is better access to my agenda; and my best guess for how to implement this is to have my agenda available on my iPhone.

I’ve tried to use mobile-org before, but just couldn’t integrate it into my workflow. Maybe that’s still the right thing, and I’ll eventually figure out how to do it. Until then, I want to just configure my agenda to automatically export to html on the web whenever I push to my central repo, and be able to review that whenever I’m out-and-about.