Good day,

After years having to deal with higher traffic = high loads on different servers and trying different things I came to the following conclusions so anyone out there can benefit from the following:

1) You must have a 2nd Hard drive so you can place the database files there (/var/lib/mysql should reside on a separate disk)
2) You must use a php accelerator such as Xcache or eaccelerator ( I prefeer xcache)
3) You must configure the parameters in httpd.conf and my.cnf according to your hardware specs ..
In httpd.conf set Timeout low (30-40)... Most people advise to turn keepalive on.. however, I disagree when you run a really heavy traffic server you should set it to OFF... you can download the script apacheload and look for apache connections in real time (I believe anything over 5 thousand connections is high traffic). It is also important to set Maxclients to a high value such as 512. Since we are handling so many connections we should also raise the conecction limit parameter in my.cnf to a higher value (F.i. max_connections=300).
Also, you should lower the values for the following parameters ( I recomend the following):
wait_timeout=60
connect_timeout=10
interactive_timeout=120
It is also important to lower CPU usage by raising the value of thread_cache_size on my.cnf. Also, key_buffer_size should be set to at least 25% of your total RAM.
4) Most high loads are consequential of heavy disk IO operations.. you must try to keep your log file rotation interval very low.. set it to rotate for file size no greater to 1GB.
5) On sata disks you MUST INCREASE read_ahead (hdparm -v /dev/sda will show your current drive settings).... hdparm -a1024 /dev/sda sets your SATA HD (/dev/sda) to use 1024 as readahead.
6) If you ran qmail or postfix on same machine you must create a ram disk and place your queue in there. You must set concurrencylocal and concurrencyremote to higher values to speed up things (F.i. local=50 and remote=100).. It is important to set backups for the queue cause once you restart your machine you will loose the queue's data.
7) You must have 127.0.0.1 as your primary name server on /etc/resolv.conf.. you can then place your secondary nameservers (theplanet's name servers)... this is important for speed.
If by looking at your server's info you notice that it is using a lot of swap memory (Anything over 1GB is bad) then you should really consider purchasing more physical RAM.
I know I posted very general info and I might have forgotten to mention other suggestions.. however please feel free to comment/ask/suggest about this.

Thanks

Forgot to mention:
You can also improve apache performance by setting modules mod_mem_cache and mod_disk_cache

Thanks this is really useful but for noobs like me, could you list it as follows (this is my current setup for my non-loading popular video site):

KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 1
MinSpareServers 5
MaxSpareServers 10
StartServers 5
MaxClients 150
MaxRequestsPerChild 1000

Please provide me with you hw specs such as CPU, hd space and type, ammount RAM mem. Also, Do you host multple sites on this server or do you only host 1 video site? if so, does your video site uses PHP+mysql? what Operating systems are you running? what is the version of your apache+php+mysql?

In order to secure your server from web attacks you should get and install mod_security for apache.

If you have a server dedicated to a single site with heavy database access you can push the MaxRequestsPerChild. However if you have many different sites with database access you would want to push MaxSpareServers higher instead and maybe lower MaxRequestsPerChild. The idea is if you have one site that has database problems, to make that one site have errors without bringing down the other sites. Either case, MaxSpareServers really should be pushed higher that 10, provably around 40 to 60.

Very sound advice. It's definitely better to lose a piece of a whole than the whole thing... You can almost always recover from it.

Hello...got some updates for you all:

1) I tried APC (php's accelerator)... very nice.. you should see a preformance gain of almost 15-20% since it lower's cpu load
2) Got 15K rpms HD SAS (performance is much better as well but not that much noticeable)
3) Since Plesk 9 version, the usage of temp filesystems for email queues is a reality