It all started with a Graceful Reboot request through WHM.

When the server restarted everything went dead except SSH access.

I attempted to start Apache through command line and it gave PHP related errors.

I attempted to access WHM to do a rebuild and got "Unable to Connect" message at both https:IP:2087 and http:IP:2086.

I disabled all PHP references in httpd.conf file and started Apache. No websites returned.

I ran /scripts/fixeverything and two html sites returned but were not serving images. One PHP site returned but obviously no php support.

I noticed in /var/log/messages that after the reboot server suffered brute force SSH attack so I added protection.

I ran /scripts/mysqlup --force with password and it installed but failed to start (Starting MySQLCouldn't find MySQL manager or server [FAILED]).

I ran /scripts/upcp --force and it gave the error text file busy.

I ran /scripts/easyapache and it said installed properly. No longer gives the PHP error. Still no websites up fully. PHP still not working.

I installed chkrootkit and ran, says no infection.

I ran netstat and killed one upcp process and all processes disappeared.

I rebooted and tried to run /scripts/upcp --force and it gives multiple errors.

I stuck my middle finger at the screen, but nothing happened.

I have a support ticket in to check the hardware, but did I miss anything?

open up /etc/my.cnf and comment out basedir, then start mysqld.

check your perl version... make sure your at about 5.8.8.

run a /scripts/upcp --force.

Just to add, proftp has also gone bye bye.




Problem 1:

root@jeffie [/scripts]# cd /etc
root@jeffie [/etc]# dir
-bash: /bin/ls: No such file or directory
root@jeffie [/etc]# dir
-bash: /bin/ls: No such file or directory
root@jeffie [/etc]# cd /etc
root@jeffie [/etc]# dir
-bash: /bin/ls: No such file or directory
root@jeffie [/etc]# edit my.cnf
-bash: /usr/bin/pico: No such file or directory
root@jeffie [/etc]# cd/
-bash: cd/: No such file or directory
root@jeffie [/etc]# cd /
root@jeffie [/]# dir
-bash: /bin/ls: No such file or directory
root@jeffie [/]#

Problem 2:
Whenever I have done /scripts/upcp --force I got file busy and after killing the upcp process in netstat it returned multiple errors and quit.

Problem 3:
I followed the advice of support ticket to do fsck and I received error:
/: recovering journal
fsck.ext3: Bad magic number in super-block while trying to re-open /
e2fsck: io manager magic bad!

I am currently FUBAR.

I have all the website data/mysql data backed up, but the downtime is killing me.

9 HOURS LATER I AM STILL WAITING FOR TECH SUPPORT TO FIX AND NOT SUGGEST THINGS THAT IRREVOCABLY DESTROY MY SERVER.

From what you have posted, the situation definitely looks bad, especially if you can not even get "ls" to work. You didn't fsck the drive while it was mounted did you?

Can you post your ticket number so I can take a look?

I found your ticket number, 4544193PLNT(for any of the other planet employees who are looking into it).

The DC technician is awaiting your OK to run some diagnostics on the drive...

If you got brute forced it shounds like they really screwed up the server for you....your best bet is probably going to be a OS reload...as if system binaries have been modified heavily, its doubtful everything will work as it should.

I was originally worried about that but every check I did showed no successful logins (not that its always accurate), and I put in an IP ban on multiple logins before most of the problems began.

I'm leaning toward http://forums.theplanet.com/index.php?show...=apache+problem

James, around 3 am I got sleepy.

Does everyone go to lunch at the same time?

Just kidding. I've always been satisfied with the service. I'd just like it if my server worked.

If anyone comes to this topic in the future...

Seriously check out:

http://forums.theplanet.com/index.php?showtopic=67117

And follow its recommendations.

Never fun to lose your server.

Thanks staff for getting it resolved.

I would avoid APF, learn iptables, you will be doing yourself a favor in the long run. I also disagree with disabling direct root login. Allowing direct root login is fine, just choose a strong password, and put sshd on a non standard port(to avoid the script kiddies).

Also the formmail scripts in cPanel... If you are using them, they were made to be secure, so there is nothing to worry about in that sense. If you aren't using it, go ahead and disable it just to be safe.

Anyone know if cphulkd(the cpanel daemon) detects brute forcing against sshd as well as cPanel?