Although I've found a few posts by searching the forums for "emic", I still have a couple of questions regarding the product from a client's perspective:
How hard was the installation process ?
Does it work "out of the box" or were there some glitches before going into production ?
General comments regarding m/cluster ?
m/cluster overview at EV1:
http://www.ev1servers.net/hosting/privater...ks/mcluster.asp
Regards,
Steffan
Full Version: Emic m/cluster experiences ?
it works nice, it needs some precautions one one of the servers crash.. but we have it deploy and we dont have any problem.
Thanks for your reply - would you mind telling me please how many nodes you have in use ? And how large is your DB in total ?
last time i check the db was about 3 gigs. and this client have 3 nodes.
1) installation process - nothing too it! Well actually it is complex but ev1 takes care of that so on your side it is easy 
2) Out of the box it is a pretty solid configuration. Ev1 already has deployed a few of them so they know what to avoid when setting it up.
3) In generally it works pretty good, especially if you are doing a lot of writes. If you are doing mostly reads mysql replication is another thing to look at, though m/cluster can scale a lot better.
If you are seriously interested also contact custom orders about it, I believe they have changed the pricing scheme for it.
2) Out of the box it is a pretty solid configuration. Ev1 already has deployed a few of them so they know what to avoid when setting it up.
3) In generally it works pretty good, especially if you are doing a lot of writes. If you are doing mostly reads mysql replication is another thing to look at, though m/cluster can scale a lot better.
If you are seriously interested also contact custom orders about it, I believe they have changed the pricing scheme for it.
QUOTE (eth00)
1) installation process - nothing too it! Well actually it is complex but ev1 takes care of that so on your side it is easy
OK
Right now, I am evaluating the product in a test environment.
3 nodes, 30GB database.
I start with only 2 nodes, let them sync - both online and well.
Now... the first request (mysql, mysqldump - doesn't matter) and one of the two nodes goes *wild*. I see up to 5 "dispatcher" processes and load goes up to the 20s. Needless to say - no more successful MySQL connections.
QUOTE
2) Out of the box it is a pretty solid configuration. Ev1 already has deployed a few of them so they know what to avoid when setting it up.
I'm especially interested in firewall settings. The emic guys mention they don't recommend any additional fw rules on the nodes but I can't leave those out.
I suspect that's one of the reasons for the trouble.
Does EV1 install the nodes with 3 NICs ?
QUOTE
3) In generally it works pretty good, especially if you are doing a lot of writes. If you are doing mostly reads mysql replication is another thing to look at, though m/cluster can scale a lot better.
Until now, we ran MySQL replication - however, this should be changed, hence the interest in m/cluster.
We generally install the nodes with two NICs ...
Has anyone tried upgrading to the latest version of m/cluster?
Has anyone tried upgrading to the latest version of m/cluster?
After working with Continuent we've found that it is usually best to install a full gigabit isolated network rather than multiple network cross-overs.
We found m/cluster to be completely unworkable. There are several bugs in the software (this was June they may have a new version by now).
Some key issues:
1. There were problems with InnoDB tables calling alter statements after a re-sync. When you have a table with millions of rows this can take a very long time.
2. Under high loads (400-600 qps, 4 node cluster), we would see some nodes go wild, run out of ram and then just churn away. To fix it required a reboot and re-sync.
3. Re-syncing a single node takes 2 nodes out of rotation. With a larger cluster this may not be significant, but with a 4 node system, you drop half of your processing power.
4. Nodes would die randomly ... nothing in the logs.
5. Problems on RHEL 4 last we used it and no 64Bit support, which is useful for DB apps to access more than 4GB of ram per process.
6. Very poor support. Had to go through EV1 to Continuent which is a slow and non-productive process.
7. No way to automatically take backups. We wanted a way to put the node in deffered standby, take a mysqldump and return it to operation. If there was a failure in sync, someone would be notified. This was not possible. We then asked about setting up replication in some way to take backups from a slave and that was not recommended either.
8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either.
Since, we've moved to MySQL 5 Cluster. Depending on your database size, you may want to look at MySQL 5 clustering. It is very, very fast for reads and writes since it is an in-memory database. MySQL 5.1 should bring disk-based clustering so database size will not be as much of a factor.
We've since abandoned m/cluster in favor of MyQL 5 Cluster and standard master/slave solutions.
Some key issues:
1. There were problems with InnoDB tables calling alter statements after a re-sync. When you have a table with millions of rows this can take a very long time.
2. Under high loads (400-600 qps, 4 node cluster), we would see some nodes go wild, run out of ram and then just churn away. To fix it required a reboot and re-sync.
3. Re-syncing a single node takes 2 nodes out of rotation. With a larger cluster this may not be significant, but with a 4 node system, you drop half of your processing power.
4. Nodes would die randomly ... nothing in the logs.
5. Problems on RHEL 4 last we used it and no 64Bit support, which is useful for DB apps to access more than 4GB of ram per process.
6. Very poor support. Had to go through EV1 to Continuent which is a slow and non-productive process.
7. No way to automatically take backups. We wanted a way to put the node in deffered standby, take a mysqldump and return it to operation. If there was a failure in sync, someone would be notified. This was not possible. We then asked about setting up replication in some way to take backups from a slave and that was not recommended either.
8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either.
Since, we've moved to MySQL 5 Cluster. Depending on your database size, you may want to look at MySQL 5 clustering. It is very, very fast for reads and writes since it is an in-memory database. MySQL 5.1 should bring disk-based clustering so database size will not be as much of a factor.
We've since abandoned m/cluster in favor of MyQL 5 Cluster and standard master/slave solutions.
"4. Nodes would die randomly ... nothing in the logs."
I'm assuming you watched the output of /usr/local/etc/eac/logs/eac-failure.log and run the script 'eacdiag.sh' to check for errors?
"8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either."
I have found that the documentation's not easily accessible for optimization/tuning parameters
Hmm ...I wonder if the 'newest' version will fix some of these 'bugs'
MySQL 5 sounds like a viable solution
I'm assuming you watched the output of /usr/local/etc/eac/logs/eac-failure.log and run the script 'eacdiag.sh' to check for errors?
"8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either."
I have found that the documentation's not easily accessible for optimization/tuning parameters
Hmm ...I wonder if the 'newest' version will fix some of these 'bugs'
MySQL 5 sounds like a viable solution
QUOTE (huck)
We found m/cluster to be completely unworkable. There are several bugs in the software (this was June they may have a new version by now).
Some key issues:
1. There were problems with InnoDB tables calling alter statements after a re-sync. When you have a table with millions of rows this can take a very long time.
2. Under high loads (400-600 qps, 4 node cluster), we would see some nodes go wild, run out of ram and then just churn away. To fix it required a reboot and re-sync.
3. Re-syncing a single node takes 2 nodes out of rotation. With a larger cluster this may not be significant, but with a 4 node system, you drop half of your processing power.
4. Nodes would die randomly ... nothing in the logs.
5. Problems on RHEL 4 last we used it and no 64Bit support, which is useful for DB apps to access more than 4GB of ram per process.
6. Very poor support. Had to go through EV1 to Continuent which is a slow and non-productive process.
7. No way to automatically take backups. We wanted a way to put the node in deffered standby, take a mysqldump and return it to operation. If there was a failure in sync, someone would be notified. This was not possible. We then asked about setting up replication in some way to take backups from a slave and that was not recommended either.
8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either.
Since, we've moved to MySQL 5 Cluster. Depending on your database size, you may want to look at MySQL 5 clustering. It is very, very fast for reads and writes since it is an in-memory database. MySQL 5.1 should bring disk-based clustering so database size will not be as much of a factor.
We've since abandoned m/cluster in favor of MyQL 5 Cluster and standard master/slave solutions.
Some key issues:
1. There were problems with InnoDB tables calling alter statements after a re-sync. When you have a table with millions of rows this can take a very long time.
2. Under high loads (400-600 qps, 4 node cluster), we would see some nodes go wild, run out of ram and then just churn away. To fix it required a reboot and re-sync.
3. Re-syncing a single node takes 2 nodes out of rotation. With a larger cluster this may not be significant, but with a 4 node system, you drop half of your processing power.
4. Nodes would die randomly ... nothing in the logs.
5. Problems on RHEL 4 last we used it and no 64Bit support, which is useful for DB apps to access more than 4GB of ram per process.
6. Very poor support. Had to go through EV1 to Continuent which is a slow and non-productive process.
7. No way to automatically take backups. We wanted a way to put the node in deffered standby, take a mysqldump and return it to operation. If there was a failure in sync, someone would be notified. This was not possible. We then asked about setting up replication in some way to take backups from a slave and that was not recommended either.
8. Tuning. To me the tuning seemed to be largely guesswork -- mostly due to lack of experience but poor documentation did not help either.
Since, we've moved to MySQL 5 Cluster. Depending on your database size, you may want to look at MySQL 5 clustering. It is very, very fast for reads and writes since it is an in-memory database. MySQL 5.1 should bring disk-based clustering so database size will not be as much of a factor.
We've since abandoned m/cluster in favor of MyQL 5 Cluster and standard master/slave solutions.
We had 2 MySQL experts, 2 Continuent Techs and 2 EV1 techs working on this cluster. Lots of expertise and eyes. One node, for whatever reason, never worked so we ditched it. The systems been running on 3 nodes for about 6 weeks, with about 4 nodes failures but we are not doing anything to the databases besides inserts, selects and deletes. The developers need to add more tables but we are simply afraid to touch the thing because a critical failure means a complete re-sync. We have a 6-7GB database, so a re-sync takes awhile.
We should have our 4 node MySQL Cluster setup within 2 weeks and will post some comments here about its peformance.
In some prelim testing with MySQL Cluster, we've found that you can run what is called an SQL node on your web server. This lets you use localhost (sockets) in PHP. We've seen many issues in PHP under high loads when connecting to a remote database. By using sockets and letting the cluster software handle the server-to-server database networking, we by-pass the connection problems often present in busy LAMP applications.
We should have our 4 node MySQL Cluster setup within 2 weeks and will post some comments here about its peformance.
In some prelim testing with MySQL Cluster, we've found that you can run what is called an SQL node on your web server. This lets you use localhost (sockets) in PHP. We've seen many issues in PHP under high loads when connecting to a remote database. By using sockets and letting the cluster software handle the server-to-server database networking, we by-pass the connection problems often present in busy LAMP applications.
This is a "lo-fi" version of our main content. To view the full version with more information, formatting and images, please click here.