-
MySQL Cluster Manager 1.2.3 Released
Posted on May 2nd, 2013 No comments
MySQL Cluster Manager 1.2.3 is now available to download from My Oracle Support.Details on the changes can be found in the MySQL Cluster Manager documentation .
-
MySQL Cluster 7.3 DMR2 – Includes MySQL Cluster Auto-Installer
Posted on April 17th, 2013 7 commentsThe second Development Milestone Release for MySQL Cluster 7.3 has just been released – download it from the development tab on this page.
Deploying a well configured cluster has just got a lot easier! Oracle have released a new auto-installer/configurator for MySQL Cluster that makes the processes extremely simple while making sure that the cluster is well configured for your application. The installer is part of the latest MySQL Cluster 7.3 Development Milestone Release and so is not yet GA but it can also be used on MySQL Cluster 7.2 (which is GA). A single command launches the web-based wizard which then steps you through configuring the cluster; to keep things even simpler, it will automatically detect the resources on your target machines and use these results together with the type of workload you specify in order to determine values for the key configuration parameters.
For a broader view of what is being released – check out this MySQL Cluster 7.3.1 DMR article.
Tutorial Video
Before going through the detailed steps, here’s a demonstration of the auto-installer in action (note that this is actually an earlier version but the idea is the same)…
Downloading and running the wizard
The software can be downloaded as part of the MySQL Cluster package from the “Development Releases” tab at MySQL Cluster DMR Download. To run on Windows, just double click setup.bat – note that if you installed from the MSI and didn’t change the install directory then this will be located somewhere like C:\Program Files (x86)\MySQL\MySQL Cluster 7.3. On Linux, just run ndb_setup from the bin folder.
If you launch the installer from a desktop environment then the first page of the wizard will automatically be opened in your web browser, if not then just browse to the URL that is displayed on the command line.
Creating your cluster
After the initial splash-screen you’re presented with the page shown in Figure 1 where you specify the list of servers that will form part of the cluster.
You also get to specify what “type” of cluster you want; if you’re experimenting for the first time then it’s probably safest to stick with “Simple testing” but for a production system you’d want to specify the application type and query profile, i.e. write-intensive.
On the next page (Figure 2), you will see the wizard attempt to auto-detect the resources on your target machines. If this fails (e.g. if you don’t have an accessible SSH server on the target hosts) then you can enter the data manually. You can also overwrite the resource-values – for example, if you don’t want the cluster to use up a big share of the memory on the target systems then just overwrite the amount of memory.It’s also on this page that you can specify where the MySQL Cluster software is stored on each of the hosts (if the defaults aren’t correct) – this should be the path to where you extracted the MySQL Cluster tar-ball file – as well as where the data (and configuration files) should be stored. You can just overwrite the values or select multiple rows and hit the “edit” button.
The following page (Figure 3) presents you with a default set of nodes (processes) and how they’ll be distributed across all of the target hosts – if you’re happy with the proposal then just advance to the next page.
Alternatively, you can add extra nodes, move nodes from one host to another (just drag and drop), delete nodes or change a node from one type to another.
On the next screen (Figure 4) you’re presented with some of the key configuration parameters that have been configured (behind the scenes, the installer sets many more) that you might want to override; if you’re happy then just progress to the next screen. If you do want to make any changes then make them here before continuing. Note that you can enable the advanced configuration option here in order to view/modify more parameters.
With the click of a button, the final screen (Figure 5) lets you deploy (copy the correct configuration settings to the hosts and create the directory structures) and start the Cluster.If you prefer or need to start the processes manually, this page also shows you the commands that you’d need to run (as well as the configuration files if you need to create them manually).
A traffic light display shows the various Cluster nodes (processes) coming into service.
Finally, you can confirm that all of the processes (nodes) are up and running (note that any API slots that don’t have MySQL Servers using them won’t show as running – that’s as expected):As always it would be great to hear some feedback especially if you’ve ideas on improving it or if you hit any problems.
-
Foreign Keys in MySQL Cluster
Posted on April 17th, 2013 17 commentsThe newly announced second MySQL Cluster 7.3 Development Milestone Release (7.3.1 m2) builds upon first DMR (7.3.0 m1) released last year which added Foreign Keys to MySQL Cluster. Foreign Keys is a feature requested by many, many people and has often been cited as the reason for not being able to replace InnoDB with MySQL Cluster when they needed the extra availability or scalability. While this functionality is not yet Generally Available (i.e. you shouldn’t deploy your application on it yet) you can can try it out by downloading the binaries from http://dev.mysql.com/downloads/cluster/ and selecting the “Development Releases” tab. Please let us have your feedback so that we know what to improve.
For a broader view of what is being released – check out this MySQL Cluster 7.3.1 DMR article.
Note that this post is an up-version of the original – and was first published with the 7.3 labs release in June 2012.
What’s a Foreign Key
The majority of readers who are already familiar with Foreign Keys can skip to the next section.
Foreign Keys are a way of implementing relationships/constraints between columns in different tables. For example, in the above figure, we want to make sure that the value of the county column in the towns table has an associated entry in the counties table. In this way, no-one can place a town in a non-existent county and similarly no one can remove a county and leave orphaned towns behind.
We refer to the towns table as the child and the counties table as the parent.
There are different categories of constraints that influence how they’re enforced when a row is updated or deleted from the parent table:
- Cascade: If a row is deleted from the parent then any rows in the child table with a matching FK value will also be deleted. Similarly for changes to the value in the parent table.
- Restrict: A row cannot be deleted from the parent table if this would break a FK constraint with the child table. Similarly for changes to the value in the parent table.
- No Action: Very similar to “Restrict” except that any events/triggers on the parent table will be executed before the constraint is enforced – giving the application writer the option to resolve any FK constraint conflicts using a stored procedure.
- Set NULL: If NULL is a permitted value for the FK column in the child table then it will be set to NULL if the associated data in the parent table is updated or deleted.
- Set Default: If there is a default value for the FK column in the child table then it will be used if the associated data in the parent table is updated or deleted. Note that this is not implemented in this version – the constraint can be added to the schema but any subsequent deletion or update to the column in the parent table will fail.
How to use Foreign Keys with MySQL Cluster
For anyone familiar with using FKs with InnoDB, a lot of this will be very similar – that’s intentional in the design of this Cluster feature. Throughout this section we will use the tables shown in the above figure.
First of all, create the tables and add some data:
mysql> CREATE DATABASE clusterdb;USE clusterdb; mysql> CREATE TABLE counties (county VARCHAR(30) NOT NULL PRIMARY KEY, country VARCHAR(30)) ENGINE=ndb; mysql> INSERT INTO counties VALUES ('Berkshire','England'),('Shropshire','England'), ('Oxfordshire','England'),('Buckinghamshire','England'); mysql> CREATE TABLE towns (town VARCHAR(30) NOT NULL PRIMARY KEY, county VARCHAR(30), INDEX county_county_index (county), CONSTRAINT county_town FOREIGN KEY (county) REFERENCES counties(county) ON DELETE RESTRICT ON UPDATE RESTRICT) ENGINE=ndb; mysql> INSERT INTO towns VALUES ('Maidenhead','Berkshire'),('Reading','Berkshire'), ('Shrewsbury','Shropshire'),('Oxford','Oxfordshire'); mysql> SELECT * FROM counties;SELECT * FROM towns; +-----------------+---------+ | county | country | +-----------------+---------+ | Shropshire | England | | Buckinghamshire | England | | Berkshire | England | | Oxfordshire | England | +-----------------+---------+ 4 rows in set (0.00 sec) +------------+-------------+ | town | county | +------------+-------------+ | Reading | Berkshire | | Shrewsbury | Shropshire | | Maidenhead | Berkshire | | Oxford | Oxfordshire | +------------+-------------+ 4 rows in set (0.00 sec)Note that the FK is actually defined as part of creating the towns table with CONSTRAINT county_town FOREIGN KEY (county) REFERENCES counties(county) ON DELETE RESTRICT ON UPDATE RESTRICT. county_town is the label assigned to the constraint (if you don’t specify one then one will be automatically created – you can see it by executing SHOW CREATE TABLE towns). The constraint is set up from the county column in the towns (child) table to the county column in the counties (parent) table.
As a first check, we can confirm that values that are not part of the constraint can be modified in the parent table – in this case we’ll move the England/Wales border:
mysql> UPDATE counties SET country='WALES' WHERE county='Shropshire'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0
Next, to illustrate the kind of user error that can be prevented, we attempt to add a town in a county that has not yet been defined – we want this to fail as it would break the constraint we created:
mysql> INSERT towns VALUES ('Cambridge','Cambridgeshire'); ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (Unknown error code) mysql> SHOW WARNINGS; +---------+------+---------------------------------------------------------------------------------------+ | Level | Code | Message | +---------+------+---------------------------------------------------------------------------------------+ | Warning | 1296 | Got error 255 'Foreign key constaint violated: No parent row found' from NDB | | Error | 1452 | Cannot add or update a child row: a foreign key constraint fails (Unknown error code) | +---------+------+---------------------------------------------------------------------------------------+ 2 rows in set (0.00 sec)We can also confirm that the constraint checking works in the opposite direction, by attempting to delete a county that has towns defined for it:
mysql> DELETE FROM counties WHERE county='Berkshire'; ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (Unknown error code) mysql> SHOW WARNINGS; +---------+------+-------------------------------------------------------------------------------------------+ | Level | Code | Message | +---------+------+-------------------------------------------------------------------------------------------+ | Warning | 1296 | Got error 256 'Foreign key constaint violated: Referenced row exists' from NDB | | Error | 1451 | Cannot delete or update a parent row: a foreign key constraint fails (Unknown error code) | +---------+------+-------------------------------------------------------------------------------------------+ 2 rows in set (0.00 sec)
It is also possible to delete FK constraints and add new ones, we do that next and at the same time we’ll switch to a CASCADE restriction. Note that adding and deleting FKs are on-line operations. In case you have forgotten the name of the constraint (or we had not explicitly named it) we can use the SHOW CREATE TABLE command to remind us:
mysql> SHOW CREATE TABLE towns; +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Table | Create Table | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | towns | CREATE TABLE `towns` ( `town` varchar(30) NOT NULL, `county` varchar(30) DEFAULT NULL, PRIMARY KEY (`town`), KEY `county_county_index` (`county`), CONSTRAINT `county_town` FOREIGN KEY(`county`) REFERENCES `counties` (`county`) ON DELETE RESTRICT ON UPDATE RESTRICT ) ENGINE=ndbcluster DEFAULT CHARSET=latin1 | +-------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ 1 row in set (0.00 sec) mysql> ALTER TABLE towns DROP FOREIGN KEY county_town; Query OK, 0 rows affected (0.47 sec) Records: 0 Duplicates: 0 Warnings: 0 mysql> ALTER TABLE towns ADD CONSTRAINT county_town FOREIGN KEY (county) REFERENCES counties(county) ON DELETE CASCADE ON UPDATE CASCADE;
The data in the tables will not have changed as part of dropping and adding constraints and so you can always DROP and the re-add constraints if you need to turn FKs off for a while:
mysql> SELECT * FROM counties;SELECT * FROM towns; +-----------------+---------+ | county | country | +-----------------+---------+ | Buckinghamshire | England | | Berkshire | England | | Oxfordshire | England | | Shropshire | WALES | +-----------------+---------+ 4 rows in set (0.00 sec) +------------+-------------+ | town | county | +------------+-------------+ | Reading | Berkshire | | Shrewsbury | Shropshire | | Maidenhead | Berkshire | | Oxford | Oxfordshire | +------------+-------------+ 4 rows in set (0.00 sec)
Now if we delete the county of Berkshire then the CASCADE option will cause all of the associated towns to be deleted too:
mysql> DELETE FROM counties WHERE county='Berkshire'; Query OK, 1 row affected (0.00 sec) mysql> SELECT * FROM counties;SELECT * FROM towns; +-----------------+---------+ | county | country | +-----------------+---------+ | Shropshire | WALES | | Buckinghamshire | England | | Oxfordshire | England | +-----------------+---------+ 3 rows in set (0.00 sec) +------------+-------------+ | town | county | +------------+-------------+ | Shrewsbury | Shropshire | | Oxford | Oxfordshire | +------------+-------------+ 2 rows in set (0.00 sec)
Note that MySQL Cluster implements FKs in the data nodes and the constraints will be enforced regardless of how the data is written (through SQL, javascript, Memcached, ClusterJ, REST or directly through the NDB API). Before illustrating this, we add back the data we have just deleted:
mysql> INSERT INTO counties VALUES ('Berkshire','England'); Query OK, 1 row affected (0.00 sec) mysql> INSERT INTO towns VALUES ('Maidenhead','Berkshire'),('Reading','Berkshire'); Query OK, 2 rows affected (0.00 sec) Records: 2 Duplicates: 0 Warnings: 0 mysql> SELECT * FROM counties;SELECT * FROM towns; +-----------------+---------+ | county | country | +-----------------+---------+ | Shropshire | WALES | | Buckinghamshire | England | | Berkshire | England | | Oxfordshire | England | +-----------------+---------+ 4 rows in set (0.00 sec) +------------+-------------+ | town | county | +------------+-------------+ | Reading | Berkshire | | Shrewsbury | Shropshire | | Maidenhead | Berkshire | | Oxford | Oxfordshire | +------------+-------------+ 4 rows in set (0.00 sec)We’ll then run a simple C++ app to delete the ‘Berkshire’ row from the counties table. The full source for the app is available here but the key part of the code is this:
static void do_change(Ndb &myNdb) { const NdbDictionary::Dictionary* myDict= myNdb.getDictionary(); const NdbDictionary::Table *countiesTable= myDict->getTable("counties"); char royalBorough[255]; royalBorough[0]=9; strcpy(royalBorough + 1, "Berkshire"); if (countiesTable == NULL) APIERROR(myDict->getNdbError()); NdbTransaction *myTransaction= myNdb.startTransaction(); if (myTransaction == NULL) APIERROR(myNdb.getNdbError()); NdbOperation *myOperation= myTransaction->getNdbOperation(countiesTable); if (myOperation == NULL) APIERROR(myTransaction->getNdbError()); std::cout << "About to delete " << royalBorough << "\n"; myOperation->deleteTuple();myOperation->equal("county", (const char *) royalBorough); if (myTransaction->execute( NdbTransaction::Commit ) == -1) APIERROR(myTransaction->getNdbError()); myNdb.closeTransaction(myTransaction); }We then run this code and confirm that the cascaded delete has worked:
billy@ws2:~/projects/fk$ ./testfk localhost:1186 About to delete Berkshire mysql> SELECT * FROM counties;SELECT * FROM towns; +-----------------+---------+ | county | country | +-----------------+---------+ | Shropshire | WALES | | Buckinghamshire | England | | Oxfordshire | England | +-----------------+---------+ 3 rows in set (0.00 sec) +------------+-------------+ | town | county | +------------+-------------+ | Shrewsbury | Shropshire | | Oxford | Oxfordshire | +------------+-------------+ 2 rows in set (0.00 sec)
Comparison with InnoDB’s Foreign Keys
As mentioned, we’ve attempted to mirror the way that InnoDB works as much as possible in the way that it implements Foreign Keys as that’s what we think will allow people to move more easily to MySQL Cluster when appropriate. There are a couple of exceptions though:
- InnoDB doesn’t support “No Action” constraints, MySQL Cluster does
- You can choose to suspend FK constraint enforcement with InnoDB using the FOREIGN_KEY_CHECKS parameter; MySQL Cluster ignores that parameter. This causes issues as some tools rely on this in order to avoid needing to define FKs and/or populate data in the correct order – examples are database restore and adding multiple FKs using MySQL Workbench. This is something to be addressed in the GA version.
- You cannot set up FKs between 2 tables where one is stored using MySQL Cluster and the other InnoDB.
- You cannot change primary keys through the NDB API which means that the MySQL Server actually has to simulate such operations by deleting and re-adding the row. If the PK in the parent table has a FK constraint on it then this causes non-ideal behaviour. With Restrict or No Action constraints, the change will result in an error. With Cascaded constraints, you’d want the rows in the child table to be updated with the new FK value but, the implicit delete of the row from the parent table would remove the associated rows from the child table and the subsequent implicit insert into the parent wouldn’t reinstate the child rows. For this reason, an attempt to add an ON UPDATE CASCADE where the parent column is a primary key will be rejected.
MySQL Workbench
While not really the subject of this post, MySQL Workbench is a great tool for database developers and one of the features is the creation, modification and reverse engineering of schemas. For example, you could import your existing schema, add a new table and then introduce a FK constraint between an existing table and your new one. Previously, WorkBench would only allow you to play with FKs for InnoDB tables; this has now been extended to allow MySQL Cluster (NDB) tables. Note: as mentioned above, there is an issue with adding multiple FK constraints with MySQL WB in this DMR – this will be addressed.Closing remarks
This DMR gives you a look at this important new feature that the MySQL engineering team have been working on. We believe that this is going to enable a whole new set of applications exploit the advantages of MySQL Cluster (on-line scalability, real-time performance, NoSQL APIs, real-time performance, on-line schema changes,…) either because:
- they want to simplify their application by pushing these referential checks down into the database – letting the developer focus more of their time on extending the application;
- because they’re stuck with 3rd party middleware that is dependent on FKs
- or that the application is already so dependent on FKs that it would be too much pain to remove them.
It is important to note that this software is not production ready and you should be using MySQL Cluster 7.2 for any live deployments.
If you are new to MySQL Cluster then here are some useful resources to get you started:
-
MySQL Cluster 7.2.12 Released
Posted on March 28th, 2013 No comments
The binary version for MySQL Cluster 7.2.12 has now been made available at http://www.mysql.com/downloads/cluster/ (GPL version) and should soon be available at https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.2.12 (compared to 7.2.10) is available from the 7.2.12 Change log.
-
MySQL Presentations from FOSDEM 2013
Posted on February 4th, 2013 2 commentsThe Free and Open source Software Developers’ European Meeting (FOSDEM) is a two-day event organized by volunteers to promote the widespread use of Free and Open Source software.I attended for the first time over the weekend and was really impressed by the number of people there, the energy and the quality of the content. The event really lives up to it’s name and is very developer-focused.
In the end, I got the opportunity to make 2 presentations. The first is a general introduction to MySQL Cluster….
The second illustrates how you can realise the benefits promised by NoSQL data stores wihtout losing the consistency and flexibility of relational databases…
-
“MySQL & Friends devroom” at FOSDEM 2013
Posted on January 18th, 2013 No commentsThe Free and Open source Software Developers’ European Meeting (FOSDEM) is a two-day event organized by volunteers to promote the widespread use of Free and Open Source software. As in previous years, there is a dedicated stream of MySQL Sessions. On Saturday (2nd Feb) evening there’s a MySQL community dinner and then we’ve a packed program from 9:15 through 17:30 on Sunday (3rd Feb).FOSDEM 2013 is a free event and there’s no requirement to pre-register – just get yourself along to Brussels.
This year I’ll be making a presentation introducing MySQL Cluster. Several of my colleagues from Oracle will also be there to present on the latest and greatest MySQL capabilities – including what’s coming in MySQL 5.6. This will be a great opportunity to politely listen to some real technical experts but an even better one to pester them for extra details – whether during the presentations or when you can grab them at other times.
Hope to see some of you there!
-
MySQL Cluster Manager 1.2.2 released
Posted on January 11th, 2013 5 comments
MySQL Cluster Manager 1.2.2 is now available to download from E-delivery and from My Oracle Support .Details on the changes can be found in the MySQL Cluster Manager documentation . Please give it a try and let me know what you think.
-
MySQL Cluster 7.2.10 Released
Posted on January 7th, 2013 No comments
The binary version for MySQL Cluster 7.2.10 has now been made available at http://www.mysql.com/downloads/cluster/ (GPL version) or https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.2.10 (compared to 7.2.9) is available from the 7.2.10 Change log.
-
MySQL Cluster Manager 1.2 – using the new features
Posted on December 13th, 2012 2 comments
Oracle have just announced that MySQL Cluster Manager 1.2 is Generally Available. For anyone not familiar with MySQL Cluster Manager – it’s a command-line management tool that makes it simpler and safer to manage your MySQL Cluster deployment – use it to create, configure, start, stop, upgrade…. your cluster.So what has changed since MCM 1.1 was released?
The first thing is that a lot of work has happened under the covers and it’s now faster, more robust and can manage larger clusters. Feature-wise you get the following (note that a couple of these were released early as part of post-GA versions of MCM 1.1):
- Automation of on-line backup and restore
- Single command to start MCM and a single-host Cluster
- Multiple clusters per site
- Single command to stop all of the MCM agents in a Cluster
- Provide more details in “show status” command
- Ability to restart “initial” the data nodes in order to wipe out the database ahead of a restore
A new version of the MySQL Cluster Manager white paper has been released that explains everything that you can do with it and also includes a tutorial for the key features; you can download it here.
Watch this video for a tutorial on using MySQL Cluster Manager, including the new features:
Using the new features
Single command to run MCM and then create and run a Cluster
A single-host cluster can very easily be created and run – an easy way to start experimenting with MySQL Cluster:
billy@black:~$ mcm/bin/mcmd –bootstrap MySQL Cluster Manager 1.2.1 started Connect to MySQL Cluster Manager by running "/home/billy/mcm-1.2.1-cluster-7.2.9_32-linux-rhel5-x86/bin/mcm" -a black.localdomain:1862 Configuring default cluster 'mycluster'... Starting default cluster 'mycluster'... Cluster 'mycluster' started successfully ndb_mgmd black.localdomain:1186 ndbd black.localdomain ndbd black.localdomain mysqld black.localdomain:3306 mysqld black.localdomain:3307 ndbapi * Connect to the database by running "/home/billy/mcm-1.2.1-cluster-7.2.9_32-linux-rhel5-x86/cluster/bin/mysql" -h black.localdomain -P 3306 -u rootYou can then connect to MCM:
billy@black:~$ mcm/bin/mcm
Or access the database itself simply by running the regular mysql client.
Extra status information
When querying the status of the processes in a Cluster, you’re now also shown the package being used for each node:
mcm> show status --process mycluster; +--------+----------+------ +---------+-----------+---------+ | NodeId | Process | Host | Status | Nodegroup | Package | +--------+----------+-------+---------+-----------+---------+ | 49 | ndb_mgmd | black | running | | 7.2.9 | | 50 | ndb_mgmd | blue | running | | 7.2.9 | | 1 | ndbd | green | running | 0 | 7.2.9 | | 2 | ndbd | brown | running | 0 | 7.2.9 | | 3 | ndbd | green | running | 1 | 7.2.9 | | 4 | ndbd | brown | running | 1 | 7.2.9 | | 51 | mysqld | black | running | | 7.2.9 | | 52 | mysqld | blue | running | | 7.2.9 | +--------+----------+-------+---------+-----------+---------+
Simplified on-line backup & restore
MySQL Cluster supports on-line backups (and the subsequent restore of that data); MySQL Cluster Manager 1.2 simplifies the process.
The database can be backed up with a single command (which in turn makes every data node in the cluster backup their data):
mcm> backup cluster mycluster;
The list command can be used to identify what backups are available in the cluster:
mcm> list backups mycluster; +----------+--------+--------+----------------------+ | BackupId | NodeId | Host | Timestamp | +----------+--------+--------+----------------------+ | 1 | 1 | green | 2012-11-31T06:41:36Z | | 1 | 2 | brown | 2012-11-31T06:41:36Z | | 1 | 3 | green | 2012-11-31T06:41:36Z | | 1 | 4 | brown | 2012-11-31T06:41:36Z | | 1 | 5 | purple | 2012-11-31T06:41:36Z | | 1 | 6 | red | 2012-11-31T06:41:36Z | | 1 | 7 | purple | 2012-11-31T06:41:36Z | | 1 | 8 | red | 2012-11-31T06:41:36Z | +----------+--------+--------+----------------------+
You may then select which of these backups you want to restore by specifying the associated BackupId when invoking the restore command:
mcm> restore cluster -I 1 mycluster;
Note that if you need to empty the database of its existing contents before performing the restore then MCM 1.2 introduces the initial option to the start cluster command which will delete all data from all MySQL Cluster tables.
Stopping all MCM agents for a site
A single command will now stop all of the agents for your site:
mcm> stop agents mysite;
Getting started with MySQL Cluster Manager
You can fetch the MCM binaries from edelivery.oracle.com and then see how to use it in the MySQL Cluster Manager white paper.
Please try it out and let us know how you get on!
-
MySQL Cluster 7.2.9 Released
Posted on November 27th, 2012 No comments
The binary version for MySQL Cluster 7.2.9 has now been made available at http://www.mysql.com/downloads/cluster/ (GPL version) or https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.2.9 (compared to 7.2.8) is available from the 7.2.9 Change log.
-
MySQL now provides support for DRBD
Posted on September 29th, 2012 2 commentsOracle has announced that it now provides support for DRBD with MySQL – this means a single point of support for the entire MySQL/DRBD/Pacemaker/Corosync/Linux stack! As part of this, we’ve released a new white paper which steps you through everything you need to do to configure this High Availability stack. The white paper provides a step-by-step guide to installing, configuring, provisioning and testing the complete MySQL and DRBD stack, including:
- MySQL Database
- DRBD kernel module and userland utilities
- Pacemaker and Corosync cluster messaging and management processes
- Oracle Linux operating system
DRBD is an extremely popular way of adding a layer of High Availability to a MySQL deployment – especially when the 99.999% availability levels delivered by MySQL Cluster isn’t needed. It can be implemented without the shared storage required for typical clustering solutions (not required by MySQL Cluster either) and so it can be a very cost effective solution for Linux environments.
Introduction to MySQL on DRBD/Pacemaker/Corosync/Oracle Linux
Figure 1 illustrates the stack that can be used to deliver a level of High Availability for the MySQL service.At the lowest level, 2 hosts are required in order to provide physical redundancy; if using a virtual environment, those 2 hosts should be on different physical machines. It is an important feature that no shared storage is required. At any point in time, the services will be active on one host and in standby mode on the other.
Pacemaker and Corosync combine to provide the clustering layer that sits between the services and the underlying hosts and operating systems. Pacemaker is responsible for starting and stopping services – ensuring that they’re running on exactly one host, delivering high availability and avoiding data corruption. Corosync provides the underlying messaging infrastructure between the nodes that enables Pacemaker to do its job; it also handles the nodes membership within the cluster and informs Pacemaker of any changes.
The core Pacemaker process does not have built in knowledge of the specific services to be managed; instead agents are used which provide a wrapper for the service-specific actions. For example, in this solution we use agents for Virtual IP Addresses, MySQL and DRBD – these are all existing agents and come packaged with Pacemaker. This white paper will demonstrate how to configure Pacemaker to use these agents to provide a High Availability stack for MySQL.
The essential services managed by Pacemaker in this configuration are DRBD, MySQL and the Virtual IP Address that applications use to connect to the active MySQL service.
DRBD synchronizes data at the block device (typically a spinning or solid state disk) – transparent to the application, database and even the file system. DRBD requires the use of a journaling file system such as ext3 or ext4. For this solution it acts in an active-standby mode – this means that at any point in time the directories being managed by DRBD are accessible for reads and writes on exactly one of the two hosts and inaccessible (even for reads) on the other. Any changes made on the active host are synchronously replicated to the standby host by DRBD.
Setting up MySQL with DRBD/Pacemaker/Corosync/Oracle Linux
Figure 2 shows the network configuration used in this paper – note that for simplicity a single network connection is used but for maximum availability in a production environment you should consider redundant network connections.A single Virtual IP (VIP) is shown in the figure (192.168.5.102) and this is the address that the application will connect to when accessing the MySQL database. Pacemaker will be responsible for migrating this between the 2 physical IP addresses.
One of the final steps in configuring Pacemaker is to add network connectivity monitoring in order to attempt to have an isolated host stop its MySQL service to avoid a “split-brain” scenario. This is achieved by having each host ping an external (not one part of the cluster) IP addresses – in this case the network router (192.168.5.1).
Figure 3 shows where the MySQL files will be stored. The MySQL binaries as well as the socket (mysql.sock) and process-id (mysql.pid) files are stored in a regular partition – independent on each host (under /var/lib/mysql/). The MySQL Server configuration file (my.cnf) and the database files (data/*) are stored in a DRBD controlled file system that at any point in time is only available on one of the two hosts – this file system is controlled by DRBD and mounted under /var/lib/mysql_drbd/.
The white paper steps through setting all of this up as well as the resources in Pacemaker/Corosync that allow detection of a problem and the failover of the storage (DRBD), database (MySQL) and the Virtual IP address used by the application to access the database – all in a coordinated way of course. As you’ll notice in Figure 4 this involves setting up quite a few entities and relationships – the paper goes through each one. -
MySQL Cluster Auto-Installer – labs release
Posted on September 29th, 2012 43 commentsDeploying a well configured cluster has just got a lot easier! Oracle have released a new auto-installer/configurator for MySQL Cluster that makes the processes extremely simple while making sure that the cluster is well configured for your application. The installer is part of MySQL Cluster 7.3 and so is not yet GA but it can also be used on MySQL Cluster 7.2. A single command launches the web-based wizard which then steps you through configuring the cluster; to keep things even simpler, it will automatically detect the resources on your target machines and use these results together with the type of workload you specify in order to determine values for the key configuration parameters.
Tutorial Video
Before going through the detailed steps, here’s a demonstration of the auto-installer in action…
Downloading and running the wizard
The software can be downloaded from MySQL Labs; just select the MySQL-Cluster-Auto-Installer build, unzip the file and then run. To run on Windows, just double click setup.bat – note that if you installed from the MSI and didn’t change the install directory then this will be located somewhere like C:\Program Files (x86)\MySQL\MySQL Cluster 7.2. On Linux, just run ndb_setup.
Creating your cluster
When you run the auto-installer it starts a small web server and then (if possible) automatically connects your web browser to it – presenting you with the first page of the wizard. If this isn’t possible (for example the server isn’t running a desktop environment), then you can connect to it remotely using the URL http://your-server-name-goes-here:8081/index.html. It may take a number of seconds to load and so please be patient. Note that the machine where you run this doesn’t need to be a host that will be included in the cluster.From the landing page, just click on the “Create new MySQL Cluster” icon to get started.
On the next page you need to specify the list of servers that will form part of the cluster. The machine where the installer is being run from needs to have ssh access to all of the cluster hosts (further, access to those machines must already have been approved from this one – if you’re uncertain, just manually connect to each one using an ssh client.By default, the wizard assumes that ssh keys have been set up (so that a password isn't needed) - if that isn't the case, just un-check the checkbox and provide your username and password.
On this page, you also get to specify what "type" of cluster you want; if you're experimenting for the first time then it's probably safest to stick with "Simple testing" but for a production system you'd want to specify the kind of application and whether it will a write-intensive application.
On the next page, you will see the wizard attempt to auto-detect the resources on your target machines. If this fails then you can enter the data manually.You can also overwrite the resource-values (for example, if you don't want the cluster to use up a big share of the memory on the target systems then just overwrite the amount of memory.
It's also on this page that you can specify where the MySQL Cluster software is stored on each of the hosts (if the defaults aren't correct) - this should be the path to where you unzipped the MySQL Cluster tar-ball/zip file - as well as where the data (and configuration files) should be stored. You can just overwrite the values or select multiple rows and hit the "edit" button.
The following page presents you with a default set of nodes (processes) and how they'll be distributed across all of the target hosts - if you're happy with the proposal then just advance to the next page. So what can you change:- Add extra nodes
- Move nodes from one host to another (just drag and drop)
- Delete nodes
- Change a node from one type to another
The diagram to the right shows an example of adding an extra MySQL Server.
On the next screen you're presented with some of the key configuration parameters that have been set (behind the scenes, the wizard sets many more) that you might want to override; if you're happy then just progress to the next screen. If you do want to make any changes then make them here before continuing. If you'd previously selected anything other than "simple" for the kind of cluster to create then you can check the "Show advanced configuration options" box in order to view/modify more parameters.
On the final screen you can review the details of the final recommended configuration and then just hit "Deploy and start cluster" and it will do just that. Depending on the complexity of the cluster, it can take a while to deploy and start everything but you're shown a progress bar together with an explanation of what stage the process is at.If for some reason you prefer or need to start the processes manually, this page also shows you the commands that you'd need to run (as well as the configuration files if you need to create them manually).
Once the wizard declares the process complete, you can check for yourself before going ahead and start your testing:
billy@black:~ $ ndb_mgm -e show Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=1 @192.168.1.106 (mysql-5.5.25 ndb-7.2.8, Nodegroup: 0, Master) id=2 @192.168.1.107 (mysql-5.5.25 ndb-7.2.8, Nodegroup: 0) [ndb_mgmd(MGM)] 2 node(s) id=49 @192.168.1.104 (mysql-5.5.25 ndb-7.2.8) id=52 @192.168.1.105 (mysql-5.5.25 ndb-7.2.8) [mysqld(API)] 9 node(s) id=50 (not connected, accepting connect from 192.168.1.104) id=51 (not connected, accepting connect from 192.168.1.104) id=53 (not connected, accepting connect from 192.168.1.105) id=54 (not connected, accepting connect from 192.168.1.105) id=55 @192.168.1.104 (mysql-5.5.25 ndb-7.2.8) id=56 @192.168.1.104 (mysql-5.5.25 ndb-7.2.8) id=57 @192.168.1.105 (mysql-5.5.25 ndb-7.2.8) id=58 @192.168.1.105 (mysql-5.5.25 ndb-7.2.8) id=59 @192.168.1.106 (mysql-5.5.25 ndb-7.2.8)
As always it would be great to hear some feedback especially if you've ideas on improving it or if you hit any problems.
-
MySQL Cluster 7.2.8 Released
Posted on September 10th, 2012 4 comments
The binary version for MySQL Cluster 7.2.8 has now been made available at http://www.mysql.com/downloads/cluster/ (GPL version) or https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.2.8 (compared to 7.2.7) is available from the 7.2.8 Change log.
-
MySQL Cluster 7.1.23 has been released
Posted on July 20th, 2012 No comments
The binary & source versions for MySQL Cluster 7.1.23 have now been made available at https://www.mysql.com/downloads/cluster/7.1.html#downloads (GPL version) or https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.1.23 (compared to 7.1.22) are available from the 7.1.23 Change log.
-
MySQL Cluster 7.2.7 released
Posted on July 20th, 2012 No comments
The binary version for MySQL Cluster 7.2.7 has now been made available at http://www.mysql.com/downloads/cluster/ (GPL version) or https://support.oracle.com/ (commercial version).A description of all of the changes (fixes) that have gone into MySQL Cluster 7.2.7 (compared to 7.2.6) are available from the 7.2.7 Change log.
-
Optimizing Performance of the MySQL Cluster Database – White Paper update
Posted on July 13th, 2012 No commentsA new version of the white paper “Guide to Optimizing Performance of the MySQL Cluster Database” has been released; download it here.This paper steps you through:
- Identifying if your application is a good fit for MySQL Cluster
- Measuring performance and identifying problem performance areas to address
- Optimizing performance:
- Access patterns
- Using Adaptive Query Localization for complex Joins
- Distribution aware applications
- Batching operations
- Schema optimizations
- Query optimization
- Parameter tuning
- Connection pools
- Multi-Threaded Data Nodes
- Alternative APIs
- Hardware enhancements
- Miscellaneous
- Scaling MySQL Cluster by Adding Nodes
As well as the kind of regular updates that are needed from time to time, this version includes the extra opportunities for optimizations that are available with MySQL Cluster 7.2 such as faster joins and engineering the threads within a multi-threaded data node.
As a reminder, I’ll be covering much of this material in an upcoming webinar.
-
MySQL Cluster : Delivering Breakthrough Performance (upcoming webinar)
Posted on July 9th, 2012 No commentsI’ll be presenting a webinar covering MySQL Cluster performance on Thursday, July 26. As always, the webinar will be free but you’ll need to register here – you’ll then also receive a link to the charts and a recording of the session after the event.The replay of this webinar is now available from here.
Here’s the agenda (hoping that I can squeeze it all in!):
- Introduction to MySQL Cluster
- Where does MySQL Cluster fit?
- Benchmarks:
- ANALYZE TABLE
- Access patterns
- AQL (fast JOINs)
- Distribution aware
- Batching
- Schema optimisations
- Connection pooling
- Multi-threaded data nodes
- High performance NoSQL APIs
- Hardware choices
- More tips
- The measure/optimise loop
- Techniques to boost performance
- Scaling out
- Other useful resources
The session starts at 9:00 am UK time / 10:00 am Central European time.
-
MySQL Cluster running on Raspberry Pi
Posted on June 1st, 2012 8 commentsI start a long weekend tonight and it’s the kids’ last day of school before their school holidays and so last night felt like the right time to play a bit. This week I received my Raspberry Pi – if you haven’t heard of it then you should take a look at the Raspberry Pi FAQ - basically it’s a ridiculously cheap ($25 or $35 if you want the top of the range model) ARM based PC that’s the size of a credit card.
A knew I had to have one to play with but what to do with it? Why not start by porting MySQL Cluster onto it? We always claim that Cluster runs on commodity hardware – surely this would be the ultimate test of that claim.
I chose the customised version of Debian – you have to copy it onto the SD memory card that acts as the storage for the Pi. Once up and running on the Pi, the first step was to increase the size of the main storage partition – it starts at about 2 Gbytes – using gparted. I then had to compile MySQL Cluster – ARM isn’t a supported platform and so there are no pre-built binaries. I needed to install a couple of packages before I could get very far:
sudo apt-get update sudo apt-get install cmake sudo apt-get install libncurses5-dev
Compilation initially got about 80% through before failing and so if you try this yourself then save yourself some time by applying the patch from this bug report before starting. The build scripts wouldn’t work but I was able to just run make…
make sudo make install
As I knew that memory was tight I tried to come up with a config.ini file that cut down on how much memory would be needed (note that 192.168.1.122 is the Raspberry Pi while 192.168.1.118 is an 8GByte Linux x86-64 PC – doesn’t seem a very fair match!):
[ndb_mgmd] hostname=192.168.1.122 NodeId=1 [ndbd default] noofreplicas=2 DataMemory=2M IndexMemory=1M DiskPageBufferMemory=4M StringMemory=5 MaxNoOfConcurrentOperations=1K MaxNoOfConcurrentTransactions=500 SharedGlobalMemory=500K LongMessageBuffer=512K MaxParallelScansPerFragment=16 MaxNoOfAttributes=100 MaxNoOfTables=20 MaxNoOfOrderedIndexes=20 [ndbd] hostname=192.168.1.122 datadir=/home/pi/mysql/ndb_data NodeId=3 [ndbd] hostname=192.168.1.118 datadir=/home/billy/my_cluster/ndbd_data NodeId=4 [mysqld] NodeId=50 [mysqld] NodeId=51 [mysqld] NodeId=52 [mysqld] NodeId=53 [mysqld] NodeId=54
Running the management node worked pretty easily but then I had problems starting the data nodes – checking how much memory I had available gave me a hint as to why!
pi@raspberrypi:~$ free -m total used free shared buffers cached Mem: 186 29 157 0 1 11 -/+ buffers/cache: 16 169 Swap: 0 0 0OK – so 157 Mbytes of memory available and no swap space, not ideal and so the next step was to use gparted again to create swap partitions on the SD card as well a massive 1Gbyte on my MySQL branded USB stick (need to persuade marketing to be a bit more generous with those). A quick edit of /etc/fstab and a restart and things were looking in better shape:
pi@raspberrypi:~$ free -m total used free shared buffers cached Mem: 186 29 157 0 1 11 -/+ buffers/cache: 16 169 Swap: 1981 0 1981Next to start up the management node and 1 data node on the Pi as well as a second data node on the Linux server “ws2″ (I want High Availability after all – OK so running the management node on the same host as a data node is a single point of failure)…
pi@raspberrypi:~/mysql$ ndb_mgmd -f conf/config.ini --configdir=/home/pi/mysql/conf/ --initial pi@raspberrypi:~/mysql$ ndbd billy@ws2:~$ ndbd -c 192.168.1.122:1186
I could then confirm that everything was up and running:
pi@raspberrypi:~$ ndb_mgm -e show Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=3 @192.168.1.122 (mysql-5.5.22 ndb-7.2.6, Nodegroup: 0, Master) id=4 @192.168.1.118 (mysql-5.5.22 ndb-7.2.6, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.1.122 (mysql-5.5.22 ndb-7.2.6) [mysqld(API)] 5 node(s) id=50 (not connected, accepting connect from any host) id=51 (not connected, accepting connect from any host) id=52 (not connected, accepting connect from any host) id=53 (not connected, accepting connect from any host) id=54 (not connected, accepting connect from any host)
Cool!
Next step is to run a MySQL Server so that I can actually test the Cluster – if I tried running that on the Pi then it caused problems (157 Mbytes of RAM doesn’t stretch as far as it used to) – on ws2:
billy@ws2:~/my_cluster$ cat conf/my.cnf [mysqld] ndbcluster datadir=/home/billy/my_cluster/mysqld_data ndb-connectstring=192.168.1.122:1186 billy@ws2:~/my_cluster$ mysqld --defaults-file=conf/my.cnf&
Check that it really has connected to the Cluster:
pi@raspberrypi:~/mysql$ ndb_mgm -e show Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=3 @192.168.1.122 (mysql-5.5.22 ndb-7.2.6, Nodegroup: 0, Master) id=4 @192.168.1.118 (mysql-5.5.22 ndb-7.2.6, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.1.122 (mysql-5.5.22 ndb-7.2.6) [mysqld(API)] 5 node(s) id=50 @192.168.1.118 (mysql-5.5.22 ndb-7.2.6) id=51 (not connected, accepting connect from any host) id=52 (not connected, accepting connect from any host) id=53 (not connected, accepting connect from any host) id=54 (not connected, accepting connect from any host)
Finally, just need to check that I can read and write data…
billy@ws2:~/my_cluster$ mysql -h 127.0.0.1 -P3306 -u root mysql> CREATE DATABASE clusterdb;USE clusterdb; Query OK, 1 row affected (0.24 sec) Database changed mysql> CREATE TABLE simples (id INT NOT NULL PRIMARY KEY) engine=ndb; 120601 13:30:20 [Note] NDB Binlog: CREATE TABLE Event: REPL$clusterdb/simples Query OK, 0 rows affected (10.13 sec) mysql> INSERT INTO simples VALUES (1),(2),(3),(4); Query OK, 4 rows affected (0.04 sec) Records: 4 Duplicates: 0 Warnings: 0 mysql> SELECT * FROM simples; +----+ | id | +----+ | 1 | | 2 | | 4 | | 3 | +----+ 4 rows in set (0.09 sec)
OK – so is there any real application to this? Well, probably not other than providing a cheap development environment – imagine scaling out to 48 data nodes, that would cost $1,680 (+ the cost of some SD cards)! More practically might be management nodes – we know that they need very few resources. As a reminder – this is not a supported platform!
-
MySQL Cluster Manager 1.1.6 released
Posted on May 29th, 2012 No comments
MySQL Cluster Manager 1.1.6 is now available to download from My Oracle Support.Details on the changes can will be added to the MySQL Cluster Manager documentation . Please give it a try and let me know what you think.
Note that if you’re not a commercial user then you can still download MySQL Cluster Manager 1.1.5 from the Oracle Software Delivery Cloud and try it out for free. Documentation is available here.
-
MySQL 5.6 Replication – webinar replay
Posted on May 29th, 2012 No commentsOn Wednesday (16th May 2012), Mat Keep and I presented on the new replication features that are previewed as part of the latest MySQL 5.6 Development Release.
The replay for that webinar (together with the chart deck) is now available from here.
In addition, there were a huge number of great questions raised and we had a couple of key engineers answering them on-line – view the Q&A transcript here.
A reminder of the topics covered in the webinar…
MySQL 5.6 delivers new replication capabilities which we will discuss in the webinar:
- High performance with Multi-Threaded Slaves and Optimized Row Based Replication
- High availability with Global Transaction Identifiers, Failover Utilities and Crash Safe Slaves & Binlog
- Data integrity with Replication Event Checksums
- Dev/Ops agility with new Replication Utilities, Time Delayed Replication and more



























