Tag: testing

On standards (and testing)

RFC 4648 defines the Base16, Base32 and Base64 encodings. Base16 (aka hex) and Base64 are widely known and used, but Base32 is an odd duck. It is rarely used, and there are several incompatible variants, of which the RFC acknowledges two: [A-Z2-7] and [0-9A-V].

One of the uses of Base32, and the reason for my interest in it, is in Google’s otpauth URI scheme for exchanging HOTP and TOTP keys. I needed a Base32 codec for my OATH library, so when a cursory search for a lightweight permissive-licensed implementation failed to turn up anything, I wrote my own.

My OATH implementation is currently deployed in an environment in which OTP keys for new users (or new OTP keys for existing users) are generated by the primary provisioning system, which passes them on to a smaller provisioning system in charge of firewalls and authentication (codenamed Nexus), which passes them on to a RADIUS server, which uses my code to validate user responses. When we transitioned from generating OTP keys manually to having the provisioning system generate them for us, we ran into trouble: some keys worked, others didn’t. It turned out to be a combination of factors:

Continue reading “On standards (and testing)” »

On testing, part III

I just got word of an embarrassing bug in OpenPAM Nummularia. The is_upper() macro, which is supposed to evaluate to true if its argument is an upper-case letter in the ASCII character set, only evaluates to true for the letter A:

#define is_upper(ch)                            \
        (ch >= 'A' && ch <= 'A')

This macro is never used directly, but it is referenced by is_letter(), which is referenced by is_pfcs(), which is used to validate paths and path-like strings, i.e. service names and module names or paths. As a consequence, OpenPAM does not support services or modules which contain an upper-case letter other than A. I never noticed because a) none of the services or modules in use on the systems I use to develop and test OpenPAM have upper-case letters in their names and b) there are no unit or regression tests for the character classification macros, nor for any code path that uses them (except openpam_readword(), which uses is_lws() and is_ws()).

The obvious course of action is to add unit tests for the character classification macros (r760) and then fix the bug (r761). In this case, complete coverage is easy to achieve since there are only 256 possible inputs for each predicate.

I have merged the fix to FreeBSD head (r262529 and r262530). Impatient users can fix their system by running the following commands:

% cd /usr/src/contrib/openpam
% svn diff -r758:762 svn://svn.openpam.org/openpam/trunk | patch
% cd /usr/src/lib/libpam/libpam
% make && make install

Unsurprisingly, writing more unit tests for OpenPAM is moving up on my TODO list. Please contact me if you have the time and inclination to help out.

On testing

Last fall, I wrote a completely new configuration parser for OpenPAM Lycopsida. Although the new parser was far more robust than the one it replaced, it was large, unwieldy, and suffered from a number of issues relating to whitespace handling, which stemmed from reusing some old code which unfortunately was thoroughly documented and therefore could not be easily modified. So I decided to rewrite it again, from scratch this time.

Then I did what I should have done last fall but didn’t: I wrote some unit tests. And of the first dozen or so tests I came up with, three failed, revealing two different bugs—one of them fairly serious.

There’s a lesson in here somewhere…