Tag: tech

Creating pkgng packages without ports

Lately, I’ve been working on expanding the scope of OpenPAM to more than just a PAM library. Specifically, I’ve added support (in a separate library) for the OATH HOTP and TOTP one-time password algorithms. In the long term, I also intend to implement PSKC and OCRA, the ultimate goal being full compliance with the OATH client and server certification profiles. Part of the reason I’m doing this is that my employer needs it, which is why the University of Oslo holds the copyright on most of the OATH code, but it is also something I’ve been wanting to do for a long time, and which I believe will greatly benefit FreeBSD.

This is a large undertaking, though. I’m not comfortable rolling a new OpenPAM release with the OATH code at this time—and I probably won’t be for quite a while. I’ve created a “nooath” branch and may roll a release from that branch in order to get the many other OpenPAM improvements into FreeBSD 10.0, but that’s a different story.

In the meantime, I need a way to test my code; not just on a development machine, but also on semi-production systems such as my desktop and my home router. Once it’s tested, I also need a way to deploy it on mission-critical systems. All these systems have one thing in common: they are binary installations, maintained with freebsd-update rather than built from source. So I need a way to install a newer version of OpenPAM without disturbing the base version.

Continue reading “Creating pkgng packages without ports”

Managing your own pkgng repository

[edit 2013-08-05: fixed a typo in the two command lines used to create the repo definition files, spotted by swills@]

Say you have your own poudriere and your own pkgng repo. You’ve set up Apache to point at your poudriere’s package directory:

<VirtualHost *>
  ServerName pkg.des.no
  ServerAdmin www@des.no
  DocumentRoot /poudriere/data/packages
  <Directory "/poudriere/data">
    Options +Indexes +SymLinksIfOwnerMatch
    IndexOptions +FancyIndexing +FoldersFirst
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

The 91amd64-default and 91i386-default directories are so named by poudriere because they contain the output of the 91amd64 and 91i386 jails, respectively, based on the default ports tree. These are details which you don’t necessarily want your clients to know (or need to know), so you create symlinks which match your clients’ ABIs:

# cd /poudriere/data/packages
# ln -s 91amd64-default freebsd:9:x86:64
# ln -s 91i386-default freebsd:9:x86:32

All you need to do on the client side now is:

# cat >/usr/local/etc/pkg.conf <<EOF
packagesite: http://pkgng.tsdutv.usit.no/freebsd:9:x86:64
EOF

Now, let’s think about this for a while. Every time you install a new machine, you have to copy or type in that pkg.conf, and while this is a pretty minimal example, your real pkg.conf could be much larger: you could have multiple repos, multiple servers with failover, etc. How about we fetch it from a central location?

# fetch -o/usr/local/etc/ http://pkg.des.no/pkg.conf

But what if it changes? Well, why not use the package system itself to distribute and maintain it?

We want to distribute our pkg.conf as a package, and since we want pkg to update it when it changes, we need to place it in a repo. We can’t stick it in the FreeBSD ports tree, and while it is possible to sneak it into the local copy of the ports tree that poudriere builds from, it’s not very convenient. So what we do is create an additional pkgng repo with only one package, which contains two pkg.conf files: one for our real pkgng repo, and one for the repo that contains our configuration package.

First, we create the contents of our package:

% mkdir des-repos
% cd des-repos
% mkdir -p usr/local/etc/pkg/repos
% cat >usr/local/etc/pkg/repos/des-packages.conf <<EOF
des-packages:
  url: http://pkg.des.no/${ABI}
EOF
% cat >usr/local/etc/pkg/repos/des-repos.conf <<EOF
des-repos:
  url: http://pkg.des.no/repos
EOF

Now we need a manifest:

% cat >+MANIFEST <<EOF
name: des-repos
version: 20130715
origin: local/des-repos
comment: Repository definitions for pkg.des.no.
arch: 
www: http://pkg.des.no/
maintainer: des@des.no
prefix: /usr/local
desc: Repository definitions for pkg.des.no.
categories: local, ports-mgmt
deps:
  pkg: { name: pkg, origin: ports-mgmt/pkg, version: 1.1 }
files:
  /usr/local/etc/pkg/repos/des-packages.conf: { uname: root, gname: wheel, perm: 0644 }
  /usr/local/etc/pkg/repos/des-repos.conf: { uname: root, gname: wheel, perm: 0644 }
EOF

Note that arch is intentionally left blank, as this package is architecture-neutral.

Once we have contents and a manifest, we can create the package file:

% pkg create -r $PWD -m $PWD
% tar tf des-repos-20130715.txz 
+COMPACT_MANIFEST
+MANIFEST
/usr/local/etc/pkg/repos/des-packages.conf
/usr/local/etc/pkg/repos/des-repos.conf

All that remains (on the server) is to create the repo:

# mkdir /poudriere/data/packages/repos
# cp des-repos-20130715.txz /poudriere/data/packages/repos
# pkg repo /poudriere/data/packages/repos
# cd /poudriere/data/packages
# ln -s repos/des-repos-20130715.txz des-repos.txz

Then, on each client (presumably including the server itself):

# rm /var/db/pkg/repo*sqlite
# rm /usr/local/etc/pkg.conf
# pkg add http://pkg.des.no/des-repos.txz
# pkg update

Tada!

Benchmark: WD Red NAS

My wife is in the market for large, cheap drives with decent performance to store sequencing data, so I ordered and tested a 2 TB Western Digital Red NAS (WD20EFRX—no link because wdc.com is broken at the moment). The Red series seems to be a halfway point between the WD Green and WD Black series: like the Green series, they have 4096-byte sectors and IntelliPower (i.e. variable rpm), but they are designed for 24×7 operation and seem to have far more consistent performance—although not quite on par with the Black series. Continue reading “Benchmark: WD Red NAS”

Progress

Amazon‘s recent acquisition of Liquavista has engendered speculation about a color Kindle. It also made me go “I told you so”.

There is an oft-repeated but apocryphal story about USPTO Commissioner Henry Ellsworth (some say Charles Duell) suggesting that the Patent Office should be shut down because “everything that can be invented has been invented”. While neither of these men ever made any such claim, a similar sentiment is surprisingly common even among technically literate people.

A while ago, I got into a discussion about emittive (CRT, LED) vs transmittive (LCD) vs reflective (eInk) display technologies. My position was that a) emittive and transmittive displays are stopgap technologies and b) high-resolution, low-latency, full-color reflective displays will be commercially available within a few years. This was immediately dismissed because, and I paraphrase, “there’s no way you’ll get the ink beads to turn fast enough”.

Chew on this for a bit.

Imagine that it’s 1880 and I tell you that “within a few years, it will be possible to travel a hundred kilometers in mere hours”, and you answer “no horse could possibly run that fast”.

Now imagine the same scenario in 1890, a few years after automobiles became commercially available.

Now imagine the same scenario in 1900, when high-end automobiles were capable of sustaining speeds of 50 km/h and above.

In the first scenario, I am looking at experiments and proofs-of-concept and hoping, fingers crossed, that a breakthrough is imminent. In the second, I am extrapolating from currently available technology and recent advances. In the third, I am simply predicting that today’s bleeding-edge technology will soon become widely available and affordable.

When I had that conversation about display technologies, black-and-white electrowetting displays were already in production, and, although I did not know this at the time, Liquavista had started shipping color EWD devkits to OEMs. They are expected to enter production this year.

I told you so.

Hurtigruten mener seg hevet over norsk lov

Hurtigruten har i flere år, med ujevne mellomrom, sendt reklame til min Gmail-adresse. Så langt i år har jeg mottatt fire såkalte nyhetsbrev fra dem.

Jeg har aldri reist med Hurtigruten. Jeg har heller aldri bedt dem om noe pristilbud, prospekt e.l. som kunne oppfattes som et ønske om å motta reklame. Jeg har aldri kontaktet dem – bortsett fra de gangene jeg har bedt dem om å slutte å sende meg reklame.

Dette er et klart brudd på Markedsføringslovens §15:

 I næringsvirksomhet er det forbudt, uten mottakerens forutgående samtykke, å rette markedsføringshenvendelser til fysiske personer ved elektroniske kommunikasjonsmetoder som tillater individuell kommunikasjon, som for eksempel elektronisk post, telefaks eller automatisert oppringningssystem (talemaskin). […] Krav om forhåndssamtykke etter første ledd gjelder heller ikke markedsføring ved elektronisk post i eksisterende kundeforhold der den næringsdrivende avtaleparten har mottatt kundens elektroniske adresse i forbindelse med salg.

Hurtigruten mener tydeligvis at de er hevet over norsk lov.

Jeg klaget dem inn til Forbrukerombudet for drøye to måneder siden, men har ikke fått noe svar. Klagen min er heller ikke journalført, hvilket i seg selv er et brudd på Offentlighetslovens §10 og dertil hørende forskrift 2008.10.17 nr 1119. Sic transit gloria mundi; Forbrukerombudet pleide å være flinke til å følge opp spam-klager, men for rundt halvannet eller to år siden sluttet de å behandle dem «på grunn av stor saksmengde».