I haven’t been able to read email sent to des@des.no or des@freebsd.org for five days, due to a series of unfortunate incidents involving dodgy power supplies and the fragility of ZFS boot in FreeBSD. Work and other duties prevented me from addressing the issue in a more timely manner, but I am now regaining control. Luckily, neither my ~30 GB IMAP spool nor any other data was lost, nor did my backup MX bounce any mail. My IMAP server is now back up with a small UFS SU+J boot / root partition instead of ZFS. I am still unable to read email, but that should be fixed within 24 hours.
I also uncovered an annoying but luckily not fatal bug in the Cyrus IMAP server. When TLS is configured, the IMAP daemon stores state for each TLS session in a DB file. If that file is corrupted, the server will start, but it will refuse any incoming IMAP or LMTP connections, and will instead spit out a stream of completely unhelpful error messages. The only recourse is to delete the TLS session state database; I set up an rc script to do that at boot time, so hopefully this won’t bite me again.
Ugh. That sounds really annoying.
Be careful with SU+J, I have heard of panic problems with it. A few are documented on the FS mailing list.
SU+J has worked beautifully for me so far…
I have never built up the courage to do a 100% ZFS system. All my machines are ZFS root, but /boot and /rescue are symlinks into a 256MB UFS /.boot filesystem so that I can recover if there’s a problem. I think I only had to so once, when a major ZFS import broke binary compat on the zpool executable. I don’t even bother with journalling /.boot, since it never gets written to.
…aaand I’m back.