Help - Search - Members - Calendar
Full Version: rDNS stats
The Planet Forums > General > Suggestions/Comments
sisterscape
OK, since I wasn't having any success getting an answer under Ensim DNS, I thought I'd try here . . .

The RS Ensim server that my site is on has been running Urchin stats since last spring.

Everything was running smoothly for many months. Then a few weeks ago, the Urchin 'no entry' domain (rDNS) stats started to increase to 25-40+%. Some adjustments were made to the settings and they returned to 19 -20%. That's the level that Urchin says is 'normal'.

But on 10/28 the 'no entry' domains were a whopping 57.5%!! Thankfully, they are now back to 20% or less.

What happened? Was that spike due to something going on at Rack Shack or some other anomaly? When Urchin finally gets installed on all RS servers, will the rDNS server(s) be able to handle the load?

I left my last site host, who also ran Urchin, because, at the time, their servers were unable to process the rDNS load. At times the 'no entry' domains were as high as 80% (I understand they finally had to get another server(s) to fix the problem.) I'm not quite sure how all this works. But I do know that stats are important to me so want to make sure that RS will be able to deliver rDNS stats consistently and accurately.

Please help solve this mystery. Thanks.
marcos
I have no idea what kind of timeout has urchin when it requests ip to domain resolution from a dns!

I assume that urchin is using the dns stated in /etc/resolv.conf.

But besides the problems with timeouts, the main problem is that there are many, many ip's without ptr! (I will bet 2 to 1 that more than 50% of ip's in the rackshack space don't resolve!). Then, it is very difficult to give you a straight answer without procesing all those ip's.. and then rechecking against the ones that could not resolve!

let me ask you... why it's so important to you to know the domain names of those that surf your web? (for me, my main consideration is the country from where they contact me).

Cheers!
Marcos
sisterscape
I don't know about the technical aspects of rDNS lookups but this is the answer from Urchin which kind of gives me an idea of the process:
QUOTE
This whole process relies on the current load on
the DNS server during the time that Urchin is
processing the logs and making requests to the DNS
server for information.  If the server is busy at
that time, Urchin will not get the same level of
return as when the server is not as busy.  You can
tell Urchin to make more attempts to resolve these
numbers from the Urchin admin GUI.  By default,
Urchin will make 5 loops (5 runs through the list
of IP addresses in the webserver log file it is
processing) and attempt to get 80% resolution -
whichever comes first.  You should increase the
number of loops to 10.  Urchin will take slightly
longer to process, but it will get a higher
resolution rate.  

Keep in mind though, if the DNS server is extremly
busy, there is not much Urchin can do to force it
to resolve more.  Increasing the loops will
definitely help though.

Increase the loops by going to Configuration;
Settings; Process Settings.  

--Urchin Support

I hope that RS will maintain sufficient resources to be able to handle the load - especially when Urchin becomes a standard feature.
QUOTE
let me ask you... why it's so important to you to know the domain names of those that surf your web? (for me, my main consideration is the country from where they contact me).

The country is important to me as well as specific educational institutions, federal and state government agencies, specific corporations etc. I am satisfied as long as the resolutions remain at about 20% - I know there will always be some domains that cannot be resolved.
marcos
the idea of loops is not bad, but I would rather control the time-out!...

let me explain a little more: no name-server has a list of all of the ip's and their resolution! (that would be quite impossible). Then, when urchin contact ns1.rackshack.net, ns1.rackshack.net needs to check which ns has the info, and then ask that ns for the info!...

There are many different machines involve in the transaction... It sounds simple, but it involves some degree of complexity, and there is a chance that if you don't give ns1 enough time to communicate with the different machines, you operation will time out.

personally, I would rather use my own dns (simple enough to setup), and then I can control the time-out not only from urchin, but from my dns!...

let me also sugest you check jdresolve... it has a very "intelligent" system to "guess" those ip's that have no ptr, based on recursing in class C, if that fails class B... not perfect, but very, very good!

I have not seen jdresolve homepage for ages... but you can download it at: http://www.jdrowell.com/static/jdresolve-0.6.1.tar.bz2

cheers!
Marcos
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.
Invision Power Board © 2001-2010 Invision Power Services, Inc.