Thinks look a bit off still, but given a couple of Mac OS X peculiarities, there are some basic guidelines to follow when debugging DNS issues on it:
- Make a long list of ISPs from around the world. Figure out (and test) their DNS servers by resolving stuff through them.
- Make a Perl script to do it automatically (I'm definetly going to do this one soon)
- Use ping and nslookup or host. The reason for this is that ping talks to lookupd (as all other applications), while nslookup talks directly to whatever DNS server you want (but defaults to the ones listed on your System Preferences).
- Get rid of Applications/Safari. Use curl or lynx to ascertain whether clients can actually resolve your site and speak HTTP 1.1.
- Nudge lookupd from time to time. It tends to cache stuff and is not always aware of changes to System Preferences or (as in my case) a local caching DNS server.
During the worst DNS problems, given that I was getting different results on my Mac and on my Linux box, to ensure lookupd wasn't getting in the way I found its process ID by doing:
$ ps -aux | grep lookupd root 321 0.0 0.1 15220 976 ?? Ss 9:16PM 0:02.64 lookupd
and then woke it up with a HUP signal.
$ sudo kill -HUP 321 Password:
This effectively forces lookupd to flush its cache and query your DNS again.
It does not, however, fix the actual problems I'm having right now.