Tonight, I went on one of my websites and noticed it was running excrutiatingly slow, so i went to my teamspeak server and got a ping of over 800ms, as opposed to my usual 50ms. I then looked into this issue a little more deeper and once I got into SSH and looked at TOP, all CPUs were at 0% idle and the load averages were all over 8. I killed all running servers on it, and it didn't bring it down any, so I rebooted it and now it seems to be better for the time being. For now, all I am running is web servers, and not starting up my game or teamspeak servers for now to see if that was the issue.

What I noticed eating up the CPUs was IOWAIT. Can someone please tell me what iowait is and what would cause it to go to almost 100% until I reboot? I tried searching some websites, but no luck.

Thanks,
-B|F

From Rob @ acunett

There's a known issue with high iowait levels and SMP machines running stock RH EL kernels. Run this in an SSH session:



That should clear it up. Add those lines to /etc/rc.local to make sure it stays that way if you ever need to reboot.

Thank you very much. I ran the script and put it in the rc.local file and seems to be running smoothly now.

Is this only for SMP machines....cause my Celly seems to be having the same problem.... ??

You can do that on any server running RHE3. I had a similar problem on one Celeron and the patch vairables fixed it for me. The latest RHE3 Kernel is meant to be much better as they implemented more sensible values by default.

Well the problem has struck again, after about 3 days of uptime. Loads are over 12 and CPUs are at 0.0% idle nonstop. The above scripts from Klaude do nothing now. They worked before, but now they don't help it in the slightest way.

Any ideas for a fix?

Which kernel do you have installed?

After a week of tweaking we have found a fix that works really good on RHE with dual xeons. We loaded up a box with 2.4.26. Turned off HT in the bios. Tried to tar several GB of info the io wait never left 0.0% and the load stayed under .50. Before we did this the io wait would max out and the load would go up to over 20.0

Going to 2.6.4 seems to resolve a lot of issues with RHE and Hyperthreading as well.

The 2.6.x kernels seem to fix this as well. I really wish RH EL supported 2.6. :/ It's the one thing keeping us from upgrading every box we find to it. We can do a 2.6 kernel for you, but we'd charge admin time to do it.

out of curiosity, how long about would the kernel upgrade take the admin and is it worth it in the long run?

You should see overall system peformance improve with a 2.6 kernel. The problem with it is it's still considered "new technology" and unsupported by most of the major commercial distros. I'm sure after a few revisions and more testing it'll be adopted by RedHat. Our Level Two support group can take care of the kernel compile and make sure all is well in usually less than an hour.

For any SM guys reading this, please review ticket #343016PLNT

I am at my wits end with SM. It is just impossible to troubleshoot an intermittent and tricky issue by means of a trouble ticket system when the usual reponse time is often in excess of 12 hours and there is no mechanism of any kind to communicate directly with one particular tech. A few emails back and forth would have probaby moved this "case" forward but the lag time and lack of continuity in "support" makes this a hopeless task.

If Klaude is reading this, please, I implore you: help me arrange to update the kernel if this is what you believe cures these high CPU and Iowait problems.