DNS over TLS in FreeBSD 12

With the arrival of OpenSSL 1.1.1, an upgraded Unbound, and some changes to the setup and init scripts, FreeBSD 12.0, currently in beta, now supports DNS over TLS out of the box.

DNS over TLS is just what it sounds like: DNS over TCP, but wrapped in a TLS session. It encrypts your requests and the server’s replies, and optionally allows you to verify the identity of the server. The advantages are protection against eavesdropping and manipulation of your DNS traffic; the drawbacks are a slight performance degradation and potential firewall traversal issues, as it runs over a non-standard port (TCP port 853) which may be blocked on some networks. Let’s take a look at how to set it up.

Basic setup

As a simple test case, let’s set up our 12.0-ALPHA10 VM to use Cloudflare’s DNS service:

# uname -r
12.0-ALPHA10
# cat >/etc/rc.conf.d/local_unbound <<EOF
local_unbound_enable="YES"
local_unbound_tls="YES"
local_unbound_forwarders="1.1.1.1@853 1.0.0.1@853"
EOF
# service local_unbound start
Performing initial setup.
destination:
/var/unbound/forward.conf created
/var/unbound/lan-zones.conf created
/var/unbound/control.conf created
/var/unbound/unbound.conf created
/etc/resolvconf.conf not modified
Original /etc/resolv.conf saved as /var/backups/resolv.conf.20181021.192629
Starting local_unbound.
Waiting for nameserver to start... good
# host www.freebsd.org
www.freebsd.org is an alias for wfe0.nyi.freebsd.org.
wfe0.nyi.freebsd.org has address 96.47.72.84
wfe0.nyi.freebsd.org has IPv6 address 2610:1c1:1:606c::50:15
wfe0.nyi.freebsd.org mail is handled by 0 .

Note that this is not a configuration you want to run in production—we will come back to this later.

Performance

The downside of DNS over TLS is the performance hit of the TCP and TLS session setup and teardown. We demonstrate this by flushing our cache and (rather crudely) measuring a cache miss and a cache hit:

# local-unbound-control reload
ok
# time host www.freebsd.org >x
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.553 total
# time host www.freebsd.org >x
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.005 total

Compare this to querying our router, a puny Soekris net5501 running Unbound 1.8.1 on FreeBSD 11.1-RELEASE:

# time host www.freebsd.org gw >x
host www.freebsd.org gw > x 0.00s user 0.00s system 0% cpu 0.232 total
# time host www.freebsd.org 192.168.144.1 >x
host www.freebsd.org gw > x 0.00s user 0.00s system 0% cpu 0.008 total

or to querying Cloudflare directly over UDP:

# time host www.freebsd.org 1.1.1.1 >x      
host www.freebsd.org 1.1.1.1 > x 0.00s user 0.00s system 0% cpu 0.272 total
# time host www.freebsd.org 1.1.1.1 >x
host www.freebsd.org 1.1.1.1 > x 0.00s user 0.00s system 0% cpu 0.013 total

(Cloudflare uses anycast routing, so it is not so unreasonable to see a cache miss during off-peak hours.)

This clearly shows the advantage of running a local caching resolver—it absorbs the cost of DNSSEC and TLS. And speaking of DNSSEC, we can separate that cost from that of TLS by reconfiguring our server without the latter:

# cat >/etc/rc.conf.d/local_unbound <<EOF
local_unbound_enable="YES"
local_unbound_tls="NO"
local_unbound_forwarders="1.1.1.1 1.0.0.1"
EOF
# service local_unbound setup
Performing initial setup.
destination:
Original /var/unbound/forward.conf saved as /var/backups/forward.conf.20181021.205328
/var/unbound/lan-zones.conf not modified
/var/unbound/control.conf not modified
Original /var/unbound/unbound.conf saved as /var/backups/unbound.conf.20181021.205328
/etc/resolvconf.conf not modified
/etc/resolv.conf not modified
# service local_unbound start
Starting local_unbound.
Waiting for nameserver to start... good
# time host www.freebsd.org >x
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.080 total
# time host www.freebsd.org >x
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.004 total

So does TLS add nearly half a second to every cache miss? Not quite, fortunately—in our previous tests, our first query was not only a cache miss but also the first query after a restart or a cache flush, resulting in a complete load and validation of the entire path from the name we queried to the root. The difference between a first and second cache miss is quite noticeable:

# time host www.freebsd.org >x 
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.546 total
# time host www.freebsd.org >x
host www.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.004 total
# time host repo.freebsd.org >x
host repo.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.168 total
# time host repo.freebsd.org >x
host repo.freebsd.org > x 0.00s user 0.00s system 0% cpu 0.004 total

Revisiting our configuration

Remember when I said that you shouldn’t run the sample configuration in production, and that I’d get back to it later? This is later.

The problem with our first configuration is that while it encrypts our DNS traffic, it does not verify the identity of the server. Our ISP could be routing all traffic to 1.1.1.1 to its own servers, logging it, and selling the information to the highest bidder. We need to tell Unbound to validate the server certificate, but there’s a catch: Unbound only knows the IP addresses of its forwarders, not their names. We have to provide it with names that will match the x509 certificates used by the servers we want to use. Let’s double-check the certificate:

# :| openssl s_client -connect 1.1.1.1:853 |& openssl x509 -noout -text |& grep DNS
DNS:*.cloudflare-dns.com, IP Address:1.1.1.1, IP Address:1.0.0.1, DNS:cloudflare-dns.com, IP Address:2606:4700:4700:0:0:0:0:1111, IP Address:2606:4700:4700:0:0:0:0:1001

This matches Cloudflare’s documentation, so let’s update our configuration:

# cat >/etc/rc.conf.d/local_unbound <<EOF
local_unbound_enable="YES"
local_unbound_tls="YES"
local_unbound_forwarders="1.1.1.1@853#cloudflare-dns.com 1.0.0.1@853#cloudflare-dns.com"
EOF
# service local_unbound setup
Performing initial setup.
destination:
Original /var/unbound/forward.conf saved as /var/backups/forward.conf.20181021.212519
/var/unbound/lan-zones.conf not modified
/var/unbound/control.conf not modified
/var/unbound/unbound.conf not modified
/etc/resolvconf.conf not modified
/etc/resolv.conf not modified
# service local_unbound restart
Stopping local_unbound.
Starting local_unbound.
Waiting for nameserver to start... good
# host www.freebsd.org
www.freebsd.org is an alias for wfe0.nyi.freebsd.org.
wfe0.nyi.freebsd.org has address 96.47.72.84
wfe0.nyi.freebsd.org has IPv6 address 2610:1c1:1:606c::50:15
wfe0.nyi.freebsd.org mail is handled by 0 .

How can we confirm that Unbound actually validates the certificate? Well, we can run Unbound in debug mode (/usr/sbin/unbound -dd -vvv) and read the debugging output… or we can confirm that it fails when given a name that does not match the certificate:

# perl -p -i -e 's/cloudflare/cloudfire/g' /etc/rc.conf.d/local_unbound
# service local_unbound setup
Performing initial setup.
destination:
Original /var/unbound/forward.conf saved as /var/backups/forward.conf.20181021.215808
/var/unbound/lan-zones.conf not modified
/var/unbound/control.conf not modified
/var/unbound/unbound.conf not modified
/etc/resolvconf.conf not modified
/etc/resolv.conf not modified
# service local_unbound restart
Stopping local_unbound.
Waiting for PIDS: 33977.
Starting local_unbound.
Waiting for nameserver to start... good
# host www.freebsd.org
Host www.freebsd.org not found: 2(SERVFAIL)

But is this really a failure to validate the certificate? Actually, no. When provided with a server name, Unbound will pass it to the server during the TLS handshake, and the server will reject the handshake if that name does not match any of its certificates. To truly verify that Unbound validates the server certificate, we have to confirm that it fails when it cannot do so. For instance, we can remove the root certificate used to sign the DNS server’s certificate from the test system’s trust store. Note that we cannot simply remove the trust store entirely, as Unbound will refuse to start if the trust store is missing or empty.

While we’re talking about trust stores, I should point out that you currently must have ca_root_nss installed for DNS over TLS to work. However, 12.0-RELEASE will ship with a pre-installed copy.

Conclusion

We’ve seen how to set up Unbound—specifically, the local_unbound service in FreeBSD 12.0—to use DNS over TLS instead of plain UDP or TCP, using Cloudflare’s public DNS service as an example. We’ve looked at the performance impact, and at how to ensure (and verify) that Unbound validates the server certificate to prevent man-in-the-middle attacks.

The question that remains is whether it is all worth it. There is undeniably a performance hit, though this may improve with TLS 1.3. More importantly, there are currently very few DNS-over-TLS providers—only one, really, since Quad9 filter their responses—and you have to weigh the advantage of encrypting your DNS traffic against the disadvantage of sending it all to a single organization. I can’t answer that question for you, but I can tell you that the parameters are evolving quickly, and if your answer is negative today, it may not remain so for long. More providers will appear. Performance will improve with TLS 1.3 and QUIC. Within a year or two, running DNS over TLS may very well become the rule rather than the experimental exception.

Twenty years

Yesterday was the twentieth anniversary of my FreeBSD commit bit, and tomorrow will be the twentieth anniversary of my first commit. I figured I’d split the difference and write a few words about it today.

My level of engagement with the FreeBSD project has varied greatly over the twenty years I’ve been a committer. There have been times when I worked on it full-time, and times when I did not touch it for months. The last few years, health issues and life events have consumed my time and sapped my energy, and my contributions have come in bursts. Commit statistics do not tell the whole story, though: even when not working on FreeBSD directly, I have worked on side projects which, like OpenPAM, may one day find their way into FreeBSD.

My contributions have not been limited to code. I was the project’s first Bugmeister; I’ve served on the Security Team for a long time, and have been both Security Officer and Deputy Security Officer; I managed the last four Core Team elections and am doing so again this year.

In return, the project has taught me much about programming and software engineering. It taught me code hygiene and the importance of clarity over cleverness; it taught me the ins and outs of revision control; it taught me the importance of good documentation, and how to write it; and it taught me good release engineering practices.

Last but not least, it has provided me with the opportunity to work with some of the best people in the field. I have the privilege today to count several of them among my friends.

For better or worse, the FreeBSD project has shaped my career and my life. It set me on the path to information security in general and IAA in particular, and opened many a door for me. I would not be where I am now without it.

I won’t pretend to be able to tell the future. I don’t know how long I will remain active in the FreeBSD project and community. It could be another twenty years; or it could be ten, or five, or less. All I know is that FreeBSD and I still have things to teach each other, and I don’t intend to call it quits any time soon.

Previously

Yes, all men

Since Susan Fowler blogged about her experience at Uber in February, the debate about sexism in tech has dominated IT and business news. Note that this debate is not new, and Fowler’s story isn’t all that different from many other stories we’ve heard before. It’s just that for once, finally, people were paying attention, and There Were Consequences. Then matters escalated in June with a string of revelations about sexism—not just discrimination, but full on sexual harassment.

Then came the apologies. Let me tell you about the apologies. The average response from a manager or venture capitalist accused of sexism went something like this:

I apologized unreservedly for my treatment of X. I realize now that my innocent jokes may have been misinterpreted. I’m actually a pretty nice guy, and X’s refusal to sleep with me had no impact whatsoever on my decision not to invest in her startup.

Then came the White Knights:

As a VC, I’m appalled to hear about my colleagues’ behavior towards women. I would like to reassure you all that Not All Men are like that. I myself am actually a pretty nice guy and completely innocent in all this.

Guys, it’s time to face the music. We have all been That Guy. We have all made sexist jokes, or laughed when others made them, or stood by silently while our male bosses, coworkers and colleagues ignored or patronized or belittled or humiliated women. We have all benefited from a system that eliminates close to 50% of our competition before the race has even started.

We are all complicit. We are all guilty.

So what do we do? Where do we go next?

First, take a deep breath, do a little soul-searching, and re-read that paragraph until any impulse, however minor, to say to yourself “OK, but not me” is gone. Yes, you too.

Next, if you’ve ever acted inappropriately towards a female coworker or friend or acquaintance, or stood by silently while others did, consider apologizing.

Third, vow to never do so again, and work hard to keep that vow. Respect the women around you as much as you respect the men. If someone around you acts or speaks inappropriately, speak up, even if there are no women present. Be proactive: make sure that women are given equal opportunity to join those career-building projects, and are included in those backstage chats where decisions are made. If you are hiring, seek out female candidates, keeping in mind that women have a tendency to underestimate their abilities and experience just as men have a tendency to overestimate them. If you are teaching, encourage and mentor female students. Reach out to them if they seem discouraged. Don’t wait until they drop out.

Open your eyes. Open your ears. Listen to the women around you. Believe them. Respect them. Be someone they can vent to and someone they can count on for support when push comes to shove.

You will slip up. When you do, apologize and vow to do better.

As a man, you are, and always have been, part of the problem. Accept it, and start being part of the solution.

Begredelig kundebehandling fra Hafslund Strøm

Jeg hadde en mildt sagt ubehagelig samtale med en selger fra Hafslund Strøm i slutten av april, som endte med at vedkommende la på. Jeg prøvde deretter å ringe både kundeservice og sentralbord uten å nå gjennom, så jeg fylte ut klageskjemaet i stedet. Nesten tre uker senere tok Hafslund endelig kontakt, og jeg sendte dem følgende beskrivelse av hendelsen:

Jeg ble oppringt av en selger ifm at jeg fortsatt står på leveringsplikt i [redigert].

Jeg prøvde å forklare for selgeren at leiligheten står tom og blokken har sentralvarme, at det ikke er noe forbruk bortsett fra håndtverkerne som pusser opp, at jeg er klar over at jeg betaler overpris men at jeg ikke er interessert i å tegne noen avtale om strømleveranse da det uansett kommer nye leietagere om noen uker.

Han ville ikke gi seg. Han fortsatte å mase om at det kostet meg 14 øre mer per kWh (eller hvor mye det nå enn var, jeg tror han ga meg to forskjellige tall i løpet av samtalen). Jeg prøvde å gjenta at leiligheten står tom, at det ikke er noe forbruk å snakke om, og spurte om han kan fortelle meg fra null ganger 14 blir? Så sluttet han å snakke om overpris og begynte i stedet å fortelle meg at jeg «må jo lese av måleren uansett» (som om det har noen betydning i forhold til valg av leverandør?) og jeg fortalte ham at joda, den ble lest av da de forrige leietagerne flyttet ut 15. mars og vil bli lest av igjen når det kommer nye leietagere om noen uker, så han byttet til barnehageonkelstemme og gjentok «du-må-le-se-av-må-le-ren-li-ke-vel» igjen og igjen til jeg kalte ham en tosk, hvorpå han la på.

Dette blir desto mer latterlig når man vet at samtalen fant sted bare noen dager før det ble satt inn ny fjernavlest måler. Så nei, jeg må ikke lese av måleren likevel.

Skriftlig beklagelse undertegnet av vedkommende selger og hans nærmeste leder kan sendes per post til:

Dag-Erling Smørgrav [redigert]

…og det bør helst ikke ta to uker.

Det har gått nå ti dager siden jeg sendte denne eposten, og rundt fire uker siden hendelsen, og jeg har fortsatt ikke fått svar, så jeg overlater saken til Internett.

Oppdatering 2017-06-29: fortsatt ikke noe svar bortsett fra en bekreftelse på at henvendelsen er mottatt.

Oppdatering 2017-09-06: nesten fire måneder uten reaksjon.

Fidelio, Act Two

As promised, here is my adaption of the second act of Beethoven‘s one and only opera Fidelio. Read the first act if you haven’t already.


Scene 1

Florestan: Shit, it’s really dark in here. It’s a good thing I’m really, really righteous and brave! I just hope Leonore is OK.


Scene 2

Fidelionore: Brr, it’s cold as balls down here.

Rocco: Sorry, Pizarro must have forgotten to pay the electricity bill. Anyway, here we are.

Fidelionore: He’s not moving!

Rocco: You think he’s dead? Nah, just asleep. Help me dig. You scared?

Fidelionore: Just cold. Sorry.

Rocco: Start digging, it’ll keep you warm.

They dig.

Fidelionore: I think he’s waking up!

Rocco: Get out of here, I need to talk to him.

Florestan: I have been imprisoned here for over two years, and surely I must know where I am and what has happened, but the audience wasn’t here, so please pretend I don’t know, and tell me who is keeping me here.

Rocco: Pizarro, and believe me, I like him about as much as you do.

Florestan: Pizarro? Shit. Send word to Sevilla, let my wife know where I am!

Rocco: Sorry, bud, no can do. I brought some wine to dull the pain of digging another man’s grave, want a drop?

Florestan: Why the hell not.

Rocco: Fidelio, bring the wine! Hey, you don’t look too good.

Florestan: Poor kid!

Fidelionore: I don’t feel too good.

Rocco: Look, it sucks that he’s going to die, but I’m only following orders.

Fidelionore: Want some stale bread? I’ve been carrying this around for days instead of grabbing a fresh piece at the breakfast table every morning.

Rocco: I am about to become complicit in an innocent man’s death, but it’s all good because I’m only following orders, plus I gave him some wine.

Florestan: I am sorry that I cannot repay you for bringing me a stale piece of bread and the dregs of your wine after you were done digging my grave.

Rocco: Okiedokie, off to tell Pizarro everything is ready.

Fidelionore: Don’t worry, it’ll all work out. Somehow. Maybe. I hope.


Scene 3

Pizarro: All done?

Rocco: All done.

Pizarro: Send the kid away and untie the prisoner while I gloat over his impending doom and make sure to let him know at whose hand it will come.

Florestan: Murderer!

Fidelionore: Murderer!

Rocco: Just following orders.

Fidelionore throws herself between Pizarro and Florestan

Fidelionore: You’ll have to kill me first!

Pizarro: Wut?

Fidelionore: I’m his wife, dumbass. Did nobody notice the hips and the tits and the fact that I’m a ducking soprano?

Pizarro: You’re his wife?

Rocco: You’re his wife?

Florestan: You’re my wife?

Pizarro: Wow, you’re really brave.

Rocco: Wow, you’re really brave.

Florestan: Wow, you’re really brave.

Pizarro: Shit, now I have to kill them both.

Fidelionore: Well, sucks to be you.

The alarm sounds.

Fidelionore: You’re saved!

Florestan: I’m saved?

Pizarro: Shit, Fernando!

Rocco: We’re saved!


Scene 4

Jaquino: Hey boss, Secretary Fernando has arrived.

Rocco: Send the guards down to, eh, ensure Pizarro gets safely up the stairs.

Fidelionore & Florestan: We’re saved!

Pizarro: I’m doomed!

Rocco: Remember, I was only following orders!


Scene 5

Fidelionore: You’re saved!

Florestan: We gonna frick!


Scene 6

Prisoners: We’re free!

Fernando: The King has sent me to free you!

Prisoners: Bonus heart-wrenching chorus!


Scene 7

Rocco: Help! Help!

Pizarro: Shut up!

Fernando: What’s all this then?

Rocco: Have mercy on Florestan—

Fernando: Florestan? Isn’t he dead?

Rocco: Only mostly dead. There’s a big difference between mostly dead and all dead. Mostly dead is slightly alive.

Fernando: Florestan is slightly alive?

Fidelionore: HE’S RIGHT IN FRONT OF YOU, YOU DUMBASS!

Rocco: …with his wife Leonore! Dressed as a man!

Marzelline: Oh shit.

Rocco: Pizarro was going to murder him!

Pizarro: And I would have gotten away with it too, if it weren’t for that meddling kid! Besides, you were helping.

Rocco: For the last time, I was only following orders!

Pizarro is arrested and taken away

Fernando: Here, Leonore, unshackle your husband.

Everybody: WE’RE SO HAPPY!

Florestan: Let this be a lesson to you all, the righteous always prevail in the end!

Fidelionore: Not to mention Tru Wuv.

Everybody: Yeah, yeah, we’re all very impressed.

Fidelionore: We gonna frick!


TEH END

Thank you, you’ve been a lovely audience.