i have a two disk setup in my server setup as



and currently the individual domain files are on hdc1 in /home2 and EVERYTHING else is on hda3. My i/o_wait times are usually under 0.5% and rarely over 3%. I have no problem with this performance.

My issue now is that my backup data is stored on hda3 along with all the mysql files. If the main drive fails, I loose all the sql data along with the backups. The backup files for my main site is 3.4gb compressed and takes 10 hours to complete daily.

Can I reliably move the mysql data files to hdc1?

Cpanel is setup to use the /home2 as the default directory for the users right now. Since hdc1 is mounted directly as /home2, if I put the mysql data folder on that drive it would have to be in the /home2 parent folder, correct?

Otherwise it would require moving data off hdc1, formatting and partitioning the drive and then remounting each partition and moving the data back to one partition as /home2 and the other as a sql folder? Then update sql config or create a symlink to the new folder.

I am also trying to find a way to switch to incremental backups to reduce the backup times and load. Creating the tarball and gzip is a huge resource hog do to the large number of images and video files I have.

another option would be to move the user data back to hda3 and leave the second drive for the incremental backups? I am no longer hosting as a business, so upgrading is out of the question due to costs.

I have a problem believing this to be true. 10hrs to compress 3.4gb of data doesn't sound right to me. What is the load of the server during the backup process? What is the memory consumption?

Out of curiosity, what does the command fdisk -l show?



On your /dev/hda3 you should have /home and thats where all your websites should reside. Then on hdc1 leave home2 or use the second drive for MySQL and Backups.

You can split up hdc into two partitions. hdc1 being 5 gigs for MySQL and hdc2 for the rest of the disk where you could pump your backups mounted as /backups.

fdisk -l output



yes it takes nearly 10 hours on a normal backup. its actually 11.4GB of files that gets compressed to 3.4GB (lots of images and video files, 450MB of DB files, other site related stuff)

load is 2.5-3.5 for all three averages, and at times spiking to 6.

memory usage is usualyl anywhere from 50-75mb free and 100-250mb swap used. i have been playing with mysql config to speed up the DB side of things so the swap usage is higher than normal.

thanks for thesuggestion on organization, but does that not put both mysql and the backups on the same physical HDD? so if it fails I dont have backup, nor do I have my DB files. I can't afford another HDD and I dont have a reliable offsite storage solution.

You should either keep your backups on another drive or move it offsite if your worried about not having them, say another server or off the pnet network.

You can move your dbs' to the second drive and as far as performance you will actually get a performance boost since Apache dont have to deal with it so much. Then again, if your dont have much MySQL usage you may find only marginal boots in performance.

If your worried about the second drive hosting your MySQL dbs what you can do is setup a cron to backup the dbs to drive 1 as well which we do independant of site backups. That way you have mysql dbs on drive 1 backed up and full site backups including mysql backups on drive 2 or offsite. That way you have 2 copies of mysql incase one is kapoot!

okay, i need some help then. how does this process sound?

1 - move data in /home2 to /hometmp
2 - unmount /home2
3 - rename /hometmp to /home2 (to maintain file links, etc for the sites)
4 - remount 2nd drive with two partitions (10gb for mysql as /sqldata, rest as /drive2)
5 - move data in /var/lib/mysql to /sqldata
6 - change /var/lib/mysql to a symlink pointing to /sqldata (see questions below)
7 - create /drive2/backup
8 - congif cpbackup to use incremental and output to /drive2/backup
9 - remove existing /backup folder and data
10 - restart httpd and mysql
11 - create /sqldata_bak and modify root cron so with a nightly "cp /sqldata /sqldata_bak"

if this works, I need command help for steps 2, 4, 6 as I dont want to loose any data.

is using a symlink for the sql data a valid and reliable solution?

otherwise I can simply edit my.cnf to use /sqldata for "datadir" and /sqldata/mysql.sock for "socket" without having to recompile anything, correct?

none of the above should effect mail delivery should it?

I would not do anything you listed without first looking at what you have. While i made the recommendation it would be dangerous to start saying yes and no to your questions without actually seeing your setup.

If you wish additional help, please contact me via PM.

PM sent, thanks!

so here is my plan. obviously I will have backups. can anyone spot any problems?

what i am wanting to do is move the site data back to the primary drive, get the mysql data files onto the secondary drive, move backups to the secondary drive. since backups are large and sql usage is high, I am hoping that this will improve disk performance. also it should limit any filesystem corruption to the specific partitions thus minimizing data loss

thoughts?

EDIT: this has been modifed with info taken from Chirpy and eth0.

<----- start ------->

- stop services

# /etc/init.d/chkservd stop
# /etc/init.d/httpd stop
# /etc/init.d/exim stop
# /etc/init.d/pure-ftpd stop
# /etc/init.d/proftpd stop
# /etc/init.d/mysql stop
# /etc/init.d/xinetd stop
# killall -9 eximstats
# /etc/init.d/cpanel stop
# /etc/init.d/crond stop
# /etc/init.d/courier-authlib stop
# /etc/init.d/courier-imap stop

- move site data to temp location on main drive (currently on second drive)


# mkdir /hometmp
# rsync -vaH /home2/* /hometmp/

- move existing backup data to temp location on main drive


# mkdir /backupold
# mv /backup /backupold
# mkdir /backup
# mkdir /backup/mysql

(recreate /backup to have spot to backup mysql (see below))

- unmount second drive

# umount /home2
# rmdir /home2

- partition second drive (120gb IDE)

# fdisk /dev/hdc
> n (new partition)
> 1 (partition 1)
> {return} (use default start)
> +10000M (make it 10GB)

> n (new partition)
> 2 (partition 2)
> {return} (use default next start)
> +50000M (make it 50GB)

> n (new partition)
> 3 (partition 3)
> {return} (use default next start)
> {return} (use rest of disk)

> w (write new partition table)

> q (quit)

(^^^ this makes mysql 10gb, backup 50gb and 60gb for other data)

- format partitions as ext3 with volume labels

# mke2fs -j -L /mysql /dev/hdc1
# mke2fs -j -L /backup /dev/hdc2
# mke2fs -j -L /data /dev/hdc3

- edit/cleanup fstab

# edit /etc/fstab

: add "/dev/hdc1 /disk2/mysql ext3 defaults 1 2"
: add "/dev/hdc2 /disk2/backup ext3 defaults 1 2"
: add "/dev/hdc3 /disk2/data ext3 defaults 1 2"

: remove any reference to "/home2"

- mount partitions

# mkdir /disk2
# mkdir /disk2/mysql
# mkdir /disk2/backup
# mkdir /disk2/data

# mount -a

- verify disk setup

# df -l

: if okay, continue

- move site data back to original name so that scripts, cpanel, etc still work without additional changes

# mv /hometmp /home2

- move sql files to new location and edit config and clean up socket file

# rsync -vaH /var/lib/mysql/ /disk2/mysql/
# edit /etc/my.cnf

: modify datadir and basedir to = /disk2/mysql
: modify pid-file to = /disk2/mysql/mysql.pid

# ln -s /disk2/mysql/mysql.sock /var/lib/mysql/mysql.sock
# rm -rf /tmp/mysql.sock
# ln -s /disk2/mysql/mysql.sock /tmp/mysql.sock

- create root cron to backup mysql back to primary drive (in case secondary drive fails and cpbackups go with it)

: make this like you want but "rsync -vaH /disk2/mysql/ /backup/mysql" every night should work

- edit WHM backup config and force backup to get current

: WHM > Configure Backup > Backup Destination = /disk2/backup

# ./scripts/cpbackup

- restart services

# /etc/init.d/crond start
# /etc/init.d/xinetd start
# /etc/init.d/cpanel start
# /etc/init.d/chkservd start

- after few days, clean up old mysql data files

# rm -rf /var/lib/mysql
# mkdir /var/lib/mysql
# chown mysql /var/lib/mysql
# service mysql restart
# ln -s /disk2/mysql/mysql.sock /var/lib/mysql/mysql.sock
# rm -rf /tmp/mysql.sock
# ln -s /disk2/mysql/mysql.sock /tmp/mysql.sock

Absolutely not! I stressed the importance of doing this with great care. And since we cannot see what is in your WHM setup, nobody can give you accurate advice.

For eg, you cannot just mv /home2 -> /home PRIMARY DRIVE if in your WHM setup LIST SITES it states that all home websites are in /home2. Because if you do that, you will loose the ability to list any sites period and all your entries in httpd.conf will be pointing to a non existing directory. Not sure what /tmphome is. Again, all this depends on your SERVER SETUP settings. Locations of /home /home2 and in WHM under LIST SITES, entries in httpd.conf etc.

I suggest you seek professional help. Thats why i suggested in via PM. This is a situation where files, directories, whm setup etc needs to be analysed first to see what you have done.

the rearrange accounts tool thinks that there is only one drive on the box, so I can not use that feature. if it worked, i have no problem using it to move the sites to /hometmp and then moving them again to the new /home2 on the primary drive.

/home2 is not the directory that your supposed to be using on the primary. The tool does not work because drive 2 contains less data then drive 1 and the logic says that if drive 1 has more data on it, create all websites on the drive that contains the less amount of data and since drive2 has less data then drive 1 it will not allow you to use the tool. And if you dont use the tool you will end up with websites pointing to non existing directories, because the migration tool also updates httpd.conf and if you have more than 20 websites and you have to do this manually you will be there all day and night.

/home2 contains more than 3x the free space than where /home is located. WHM is setup to use /home2 as Default Home Directory and "home" is the Home Directory Prefix.

I have three sites and I can easily manually update httpd.conf to point to the new location. Is this an issue since I would be using the same folder in the end (/home2) its just on a different drive? nothing in httpd.conf refers to the disk/mount itself, only folders.

I admit that I do not know if exim, whm, cpanel or others use references to the physical drive instead of simple references to the folders (irregardless of drive they are on)

my /hometmp is a folder on the primary drive where i move all the existing site data to so i can unmount the secondary drive (which is mounted as /home2). Since /home2 exists already, I can not create a /home2 folder on the primary drive without these two steps.

Once I unmount the secondary drive, i can then rename /hometmp to /home2 so nothing has really changed except for which drive /home2 is sitting on.

Does WHM/Cpanel or other software refer to the phsyical drive rather than just the folder when operating? If so, that seems somewhat rediculous, but hey, I am not a programmer so what do I know....

i have editted the post above after reviewing some info by chirpy and eth0

just to keep with my rambling on today, i suppose that it would be more efficient to have a single partition in terms of space utilization, but i get worried about a filesystem issue that could wipe the whole drive in this case, rather than be limited to one of the partitions.

also, i did some testing regarding the best way to backup the mysql data to the primary drive (remember the backups will be on the same HDD as mysql data after the changes happen). Using mysqldump with --all-databases option nearly cripples the server. i think that rsync'ing the files directly is the best performance option, especially since this is a secondary backup.

well i did the above work today. went off without a hitch. i did however stick to a single partition on the second drive for simplicity. so mysql is working great and backups are working great.

i/o wait is down, loads are down, disk space use is not as balanced but that is okay with me given the performance results so far. hopefully in a few days it will still be going the same.

more rambling today. things are working great, performance is way up, response times are better, everything seems better. kind of bummed i didnt do all this a long time ago.