We have an image-intensive site that's currently around 64GB. That also includes a mySQL database. cpBackup completes about every other day. Since images don't compress much, the backup consumes the same amount of space. We have a dedicated backup drive, 147GB.
Even after some pretty rigorous house-cleaning, the site is still large. And it will only get larger. That's the nature of the business run from the site. Could someone please recommend a reliable backup strategy outside of cpBackup that I can use for a site this huge? I have root access, I'm comfortable with SSH, so would rsync, scp, the like suffice? What about stressing out the home server?
I've googled and googled and googled, but I have yet to find one concrete, proven methodology for performing GINORMOUS SITE backups. I know more space is warranted, sure, but how much? Or can I just set up a job to backup to the site owner's computer? Any advice or direction you could offer from your experience would be greatly appreciated. Thanks!!!!
Full Version: cpbackup of LARGE site unreliable. Help!
QUOTE (nennikers @ Jan 7 2008, 10:11 AM) 
We have an image-intensive site that's currently around 64GB. That also includes a mySQL database. cpBackup completes about every other day. Since images don't compress much, the backup consumes the same amount of space. We have a dedicated backup drive, 147GB.
Even after some pretty rigorous house-cleaning, the site is still large. And it will only get larger. That's the nature of the business run from the site. Could someone please recommend a reliable backup strategy outside of cpBackup that I can use for a site this huge? I have root access, I'm comfortable with SSH, so would rsync, scp, the like suffice? What about stressing out the home server?
I've googled and googled and googled, but I have yet to find one concrete, proven methodology for performing GINORMOUS SITE backups. I know more space is warranted, sure, but how much? Or can I just set up a job to backup to the site owner's computer? Any advice or direction you could offer from your experience would be greatly appreciated. Thanks!!!!
Even after some pretty rigorous house-cleaning, the site is still large. And it will only get larger. That's the nature of the business run from the site. Could someone please recommend a reliable backup strategy outside of cpBackup that I can use for a site this huge? I have root access, I'm comfortable with SSH, so would rsync, scp, the like suffice? What about stressing out the home server?
I've googled and googled and googled, but I have yet to find one concrete, proven methodology for performing GINORMOUS SITE backups. I know more space is warranted, sure, but how much? Or can I just set up a job to backup to the site owner's computer? Any advice or direction you could offer from your experience would be greatly appreciated. Thanks!!!!
Whats your budget on this?
You could look into TP's dedicated backup servers. They come with the R1Soft software.
http://www.theplanet.com/hosting%2Dproducts/backup/
disksync is good... but again its costly. At least only the changed files get backed up instead of everything.
Do you have cpanel's backup set to incremental?
I sure wouldn't try to tar.gz all that, but I would have thought that incremental (which as far as I can tell is just rsyncing it over to the backup directory) would have worked.
I'd put in a bugzilla report over at cpanel.net if you're using the incremental and it's erroring out.
If a crash occurred, do they have a broad enough pipe to get 64 GB back to the server in a reasonable amount of time ???
I sure wouldn't try to tar.gz all that, but I would have thought that incremental (which as far as I can tell is just rsyncing it over to the backup directory) would have worked.
I'd put in a bugzilla report over at cpanel.net if you're using the incremental and it's erroring out.
QUOTE (nennikers @ Jan 7 2008, 11:11 AM)
Or can I just set up a job to backup to the site owner's computer?
If a crash occurred, do they have a broad enough pipe to get 64 GB back to the server in a reasonable amount of time ???
From a technical point of view, I like the way the CDP servers do their backups. Incremental at the block level. You can set the incremental down to 5 minutes and the load gets spread out.
This is purely from a technical viewpoint and not a business viewpoint.
This is purely from a technical viewpoint and not a business viewpoint.
QUOTE (TP-TDurden @ Jan 9 2008, 07:45 PM)
From a technical point of view, I like the way the CDP servers do their backups. Incremental at the block level. You can set the incremental down to 5 minutes and the load gets spread out.
This is purely from a technical viewpoint and not a business viewpoint.
This is purely from a technical viewpoint and not a business viewpoint.
Not only that, but you can also set the CDP server to backup every PC or on-site server you have in your office, if you really wanted..
I'm a fan of CDP as well. If anyone is curious about the technical side of it... http://www.r1soft.com/documentation.html.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.