I was troubleshooting the one and just could NOT find it. Then I noticed it was also failing on the other and I knew something very bad was going on.

I just found this on CPanel's forums:





Anyone else?

If they hosed everyone that's going to be horrifying.

I just talked to a tech who got me fixed up right away (thanks . Apparently there was a bad update.

He said something about changing my resolv.conf to point to a server other than myself so 'up2date -u bind-libs' could run. Then he changed resolv.conf back.

I have this problem also.

The fix should be posted here, people are going to come looking for it!

Quibbler

The solution for me was to :-

backup /etc/resolv.conf

add to the top of resolv.conf the ip of a working theplanet nameserver, I used :-



Save resolv.conf

Then run



If completes the update successfully



Problem fixed. Use at your own risk!

Quibbler

Thanks! That worked.

How do I prevent this from occuring again? Is there a ServerMatrix service for DNS redundancy, similar to DynDNS?

This worked for me, too. Just found about the fix after it got fixed. Had submitted a ticket and Support got me backup in less than 5 minutes!!!!!

Thanks to ThePlanet/Servermatrix

good question. i was in a full-on panic and started signing up for easydns (20/yr) becaues i could not figure it out.

This should be sticky as I am experiencing (was experiencing the same problem). The funny thing was i was experiencing a brute force attack just before this update happed so my log looked pretty scary...

Thanks for the fix.

Wow - I had this too...

The fix is not working for me, but I also updated cPanel and did some other things thinking that cPanel just fsck'ed itself - again...

Fixed - in my attempts to fix it, I changed the permissions and forgot to change them back - mea culpa

This wasn't only a cpanel issue, but yes, cpanel servers were affected. I will get with our internal services guys to see if we can't find a way to prevent this from happening again.

On a side note, I would highly recommend keeping at least 1 external resolver in your /etc/resolv.conf to ensure that you can resolve hosts if your named server goes down. This way, 'updates' can be pushed out at a later time. If you do not have an external resolver listed, and your named service is down, you will also not be able to recieve updates from the RHN servers.

You can sign up for rollernet.us or something similar, which is free.

The issue is really two fold, and while non CPanel users might have seen it, that's somewhat of an edge case.

The bind rpm has a dependency list as such:



The dependency lists looks OK on the surface, but notice how bind-utils is a dependency, but none of the other bind packages are depeancies. For the bind-libs package, the bind rpm has files bind-libs PROVIDES as dependencies, but not the package itself.

Here's where the problem pops up: the files provided by bind-libs have been updated, but the version numbers on the libraries have not changed. This means that the old bind-libs package solves the dependencies for libdns, libisc, libiscfg, etc. Unfortunately, the new set of libraries breaks binary compatibility with the bind package. (something which is not supposed to happen) Both packages have to be updated at the same time.

If you run, say, 'up2date -u bind' up2date will update bind for you. It will also most certainly update bind-utils for you as well, but bind-libs will not be updated. This is because of the way the dependencies are built: the bind-utils dependency will force a package version check, up2date will see that the installed version of bind-utils is older than the available version and update. There is no bind-libs dependency so there's never a version check. Running 'up2date -u' works around this problem by forcing up2date to do a version check on all of the installed packages, so it would see the old bind-libs package installed and update it accordingly.

Now take a look what how CPanel runs up2date via /scripts/sysup (YMMV, I grabbed this from ps(1) output on a test system):



Oops. CPanel specified bind, bind-devel and bind-utils. No bind-libs, so no version check on that package is ever done and since the libraries already exist the dependency is considered solved.

We believe that we have implemented a good work around; we've "scheduled" all 4 of the bind packages for every RHEL3 system that has bind installed on it. That should force up2date to update bind properly the next time the system checks in (once every couple of hours IIRC). Get ought to get a working bind-libs out to all the systems that need it, and cause the other systems to update all of the bind packages, hopefully in the right order.

Knowing that things like this happen from time to time (blame it on cPanel if you want), I have disabled automatic updates on all our servers. I rather sit down once a month a initiate all the updates myself

When I read about this issue today, I thought I would be proactive and do the Bind update manually, before cPanel does it for me one of theses days. Little did I know that the instructions above works for fixing the problem, rather than be a complete set of update instructions in itself. I ended up with a mismatch of RPM versions and Bind not wanting to run because of that.

I received the following errors:


Should you wish to run the Bind update yourself and do not have the cPanel induced problem yet, I suggest the following:

1) Make sure you have a valid setup in your /etc resolv.conf as explained above

2) Update both Bind and it libraries:


I hope that helps someone in the same situation.

With that as the fist nameserver entry, Exim suddenly gave us "unroutable domain" errors. I moved the main server IP back into the first position, and all was fine again.

Does this mean the TP name server 70.87.7.70 does not allow recursion for our servers?

I had the same problem i to have moved it to the bottom of the list, well see how it goes.

Thanks this allowed me to fix my server before my techs responded!

Is their anyway to ping the status of named service?

I have monitoring scripts that ping http, smtp, pop3 etc what port does named run on? or how do I find out?

Cheers

Andrew

DNS uses port 53...

I thought that and thats what port I use to monitor DNS but I got no failure notifications. :shock:

Time to investigate

Thanks

Andrew

Kewl.

Browsed around, looks like there are many others. Couldn't find a comparison matrix between all these Secondary DNS services, though...

Could you give me pointers as to criteria for choosing a 2DNS server?

Thanks

That's an authorative name server. It does not do recursion.