MySQL Cluster 7.1.5 binaries released

The binary version for MySQL Cluster 7.1.5 has now been made available at http://www.mysql.com/downloads/cluster/

A description of all of the changes (fixes) that have gone into MySQL Cluster 7.1.5 (compared to 7.1.4) can be found in the MySQL Cluster 7.1.5 Change Log.

Upcoming webinar: MySQL Cluster deployment best practices

On Wednesday 25 August, Johan Anderson and Mat keep will be presenting a free webinar on the best practices for deploying MySQL Cluster. If you’d like to attend then just register here.

Johan has years of MySQL Cluster professional services experience – and so if you’re interested in deploying a product ready MySQL Cluster database then this is a must-view session. Even if you can’t make it live, register anyway and you’ll be sent a link to the recording and charts afterwards.

Content

An invaluable session for those who are about to, or who already have, deployed MySQL Cluster. Delivered by the lead MySQL Cluster professional services consultant in Oracle, this session will present best practices on deploying MySQL Cluster in order to accelerate time to service with the highest levels of availability and performance to support your most critical web and telecoms applications.

The session will cover:

WHEN

Install, configure and run MySQL Cluster – demo video

There is a new video available: Demonstration of installing, configuring and running MySQL Cluster (LINUX) to accompany the MySQL Cluster Quick Start guides. The Flash video video lasts for about 7 minutes.

If you aren’t able to view Flash on your device then a (poorer quality) version is included here – watch the Flash version if you’re able to!

Replacing MEMORY storage engine with MySQL Cluster

Many people use the MySQL MEMORY storage engine for applications where they don’t need their data to be highly available or even survive restarts. Where this works best is for applications that are dominated by reads and (obviously) where the data doesn’t need to survive any kinds of problems (or where it can be recovered from another source). Typical applications are :

While great for some applications (at least initially), as the application requirements start to grow (need for transactions, data persistence/durability, increasing number of write operations, need to scale) then either you need to migrate away from MEMORY or implement more and more data management functionality needs in your application.

Why not consider using MySQL Cluster instead? While the headline Cluster configurations may scale out horizontally and provide 99.999+% availability, it is simple to configure out those features to provide a pure-memory, single host database as described in “How can a database be in-memory and durable at the same time?“. What’s more if/when you need to add extra capabilities such as ACID transactions, scalability and high availability then those features are all available to you.

If you’re constrained to a single server and your application only reads data then there is a performance impact in using MySQL Cluster (in benchmarks, throughput was 3x higher for MEMORY) but remember that you can scale out MySQL Cluster to increase performance and for models with modest write rates MySQL Cluster provides a massive performance boost even when constrained to 1 server (as shown in the graph at the head of this post, Cluster delivers a 30x improvement performance improvement when running a Sysbench benchmark – together with a 3x reduction in latency). The biggest single reason for the massive performance difference is that MySQL Cluster has row-based locks whereas MEMORY uses table-level locks.

For a more detailed comparison, take a look at the new “Scaling Web Services with MySQL Cluster: An Alternative to the MySQL Memory Storage Engine” white paper and/or listen to the associated webinar.

Migrating Data from The Memory Storage Engine to MySQL Cluster

As a starting point, it is important to understand that the MySQL Server (mysqld) binary that is provided as part of MySQL Community Edition or MySQL Enterprise Server is not suitable for use with MySQL Cluster.  Therefore it is necessary to migrate your database to an instance of mysqld that is packaged with the MySQL Cluster software.

This post looks at how to migrate your data from a ‘regular’ MySQL Server to MySQL Cluster. Although MySQL Cluster is typically run over multiple servers to provide High Availability (HA), for this example migration it is run on a single server (the logic being that we’re replacing a MEMORY database that has no HA requirements). If you want to move to a HA configuration then the main change is that the MySQL Cluster configuration should include more nodes (processes) and they should be spread across multiple physical hosts. The example actually goes a step further in compromising HA and turns off the checkpointing of data to disk for the migrated tables – getting closer to the behavior of MEMORY tables.

For this example, there are assumed to be 2 MySQL installations on the server – the original binaries in <mysql-bin> and the ones for MySQL Cluster in <mysql-cluster-bin>.

Data in MyISAM and InnoDB tables (including system data such as user privileges and stored procedures) will appear in the new MySQL Server automatically.  By default the contents (but not schema) for the MEMORY tables will be lost. If it is important to keep that MEMORY data then use the mysqldump command to make a copy:

[my_db]$ <mysql-bin>/mysqldump -h 127.0.0.1 -P3306 -u root --no-create-db --no-create-info --result-file=./data.txt --databases clusterdb > memory_data.txt

Create a new configuration file for the MySQL Cluster deployment:

my_db/conf/config.ini:

[ndbd default]
noofreplicas=1
datadir=/home/billy/mysql/my_db/data
DataMemory=1000M
IndexMemory=10M

[ndbd]
hostname=localhost

[ndb_mgmd]
hostname=localhost
datadir=/home/billy/mysql/my_db/data

[mysqld]

Start up the management node and data node for the Cluster:

[my_db]$ <mysql-cluster-bin>/ndb_mgmd -f conf/config.ini --configdir=./conf/ --initial
[my_db]$ <mysql-cluster-bin>/ndbd -c 127.0.0.1:1186

Wait until the nodes are started:

[my_db]$ <mysql-cluster-bin>/ndb_mgm -e show

Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------

[ndbd(NDB)]	 1 node(s)
id=1	 @127.0.0.1	 (mysql-5.1.44 ndb-7.1.3, Nodegroup: 0, Master)

[ndb_mgmd(MGM)] 1 node(s)
id=2	 @127.0.0.1	(mysql-5.1.44 ndb-7.1.3)

[mysqld(API)]	 1 node(s)
id=3 (not connected, accepting connect from any host)

Edit the configuration file (from the original MySQL Server) by adding the highlighted rows:

my_db/conf/my.cnf:

[mysqld]
ndbcluster
datadir=/home/billy/mysql/my_db/data
basedir=/usr/local/mysqlc # The MySQL Cluster installation, not the original

Stop the original MySQL Server:

[my_db]$ <mysql-bin>/mysqladmin -u root -h 127.0.0.1 -P 3306 shutdown

Start the new MySQL Server:

[my_db]$ <mysql-cluster-bin>/mysqld --defaults-file=conf/my.cnf &

Make sure that the original database is upgraded so that the MySQL Cluster version of mysqld can properly use it and you get all of the benefits (for example, real-time reporting with ndbinfo):

[my_db]$ <mysql-cluster-bin>/mysql_upgrade

Convert the MEMORY tables into MySQL Cluster ones (engine=ndb) with checkpointing to disk (logging) turned off:

[my_db]$ <mysql-cluster-bin>/ mysql -u root -h 127.0.0.1 -P 3306

mysql> SET ndb_table_no_logging=1;
mysql> ALTER TABLE tab1 engine=ndb;
mysql> ALTER TABLE tab2 engine=ndb;
mysql> ALTER TABLE tab3 engine=ndb;
mysql> SET ndb_table_no_logging=0;

If the contents of the MEMORY tables were backed up then that data can optionally be loaded at this point (original MyISAM and InnoDB table data will already be populated):

mysql> SOURCE memory_data.txt;

Finally, if there is any intent to use MySQL replication with the new version of the database then check that each of the tables has a primary key defined. If there is a table without a primary key then either make one or more of the existing columns be the primary key or add a new, auto-incrementing one:

mysql> ALTER TABLE tab1 ADD COLUMN id BIGINT AUTO_INCREMENT PRIMARY KEY;

This procedure involves loss of service while the data is migrated (it assumes that there is not enough memory available on the server for both databases to exist in parallel). If there is extra capacity available (at least during the migration) then it is possible to avoid the downtime by using replication.

In many cases, the migration would not be from a single MySQL Server to a single-server MySQL Cluster deployment, but instead from a pool of independent MySQL Servers (where the application shards data across them) to a multi-server MySQL deployment. This allows the application to stop sharding the data as all mysqld instances in the Cluster have visibility of all of the data. In this scenario, if the data from the original MySQL Servers needs to be maintained then a mysqldump would be performed on each of those Servers (for all tables) and the data loaded into any of the MySQL Servers in the MySQL Cluster deployment.

Free webinar – Scaling web apps with MySQL (an alternative to the MEMORY storage engine)

Mat Keep and I will be presenting this free webinar on Wednesday 14 July.

The MEMORY storage engine has been widely adopted by MySQL users to provide near-instant responsiveness with use cases such as caching and web session management. As these services evolve to support more users, so the scalability and availability demands can start to exceed the capabilities of the MEMORY storage engine.

The MySQL Cluster database, which itself can be implemented as a MySQL storage engine, is a viable alternative to address these evolving web service demands. MySQL Cluster can be configured and run in the same way as the MEMORY storage engine (ie on a single host with no replication and no persistence). As web services evolve, any of these attributes can then be added in any combination to deliver higher levels of scalability, availability and database functionality, especially for those workloads which predominately access data by the primary key.

As always, the webinar is free of charge but you will need to register here.

Time:

• Wed, Jul 14: 06:00 Hawaii time
• Wed, Jul 14:  09:00 Pacific time (America)
• Wed, Jul 14: 10:00 Mountain time (America)
• Wed, Jul 14: 11:00 Central time (America)
• Wed, Jul 14: 12:00 Eastern time (America)
• Wed, Jul 14: 16:00 UTC
• Wed, Jul 14: 17:00 Western European time
• Wed, Jul 14: 18:00 Central European time
• Wed, Jul 14: 19:00 Eastern European time

If you can’t make the live webinar then register anyway and you’ll get sent a link to the recording after the event.

MySQL Workbench 5.2 goes GA – partial support for MySQL Cluster

Configure MySQL Server nodes for MySQL Cluster

The new version of MySQL Workbench (5.2.25) has just gone GA – see the Workbench BLOG for details.

So what’s the relevance to MySQL Cluster? If you have a Cluster that uses MySQL Servers to provide SQL access then you can now use MySQL Workbench to manage those nodes:

• Start & stop the mysqld processes
• Configure the per-mysqld configuration data held in my.cnf or my.ini

The reason that I describe the support as ‘partial’ is that these MySQL Servers are treated as independent entities (no concept of them being part of a Cluster) and there is currently no way to use it to configure or manage the other Cluster processes (data and management nodes). Having said that, what is there provides a lot of value and Workbench is designed to be very extensible  and so hopefully there can be further MySQL Cluster support in the future.

View MySQL Cluster status variables

In addition to MySQL Cluster-specific configuration parameters, you can also access the Cluster-specific status variables (these are the ones starting with ndb).

While I’ve focussed on what’s unique to MySQL Cluster, you can of course use the other Workbench features with MySQL Cluster – for example:

• Creating (or reverse-engineering) your data model
• Define your schema
• View/write data to your tables
• Create your SQL queries

Using Syslog with MySQL Cluster

By default, MySQL Cluster sends log data to a file but you can also send it to the console or to Syslog; this article explains how to send it to Syslog. The example given here is for LINUX.

In this example, I’ll use the “user” syslog facility name and so the first step is to make sure that syslog is configured to route those messages. If this hasn’t already been configured then add the following lines to /etc/rsyslog.conf:

# Log user messages to local files
user.*    /var/log/user

For the changes to take effect, restart the syslog service:

[root@ws1 etc]# service rsyslog restart
Shutting down system logger:                               [  OK  ]
Starting system logger:                                    [  OK  ]

Note that you should make those changes as root.

Still as root, start up a stream of  any additions to the new log file:

[root@ws1 etc]# tail -f /var/log/user

To tell Cluster to use Syslog, add this line into the [ndb_mgmd] section in config.ini:

LogDestination=SYSLOG:facility=user

and then start up your Cluster as normal.

You should now be able to see that MySQL Cluster information is being logged to /var/log/user.

You can adjust how much information is logged either through the config file or from the ndb_mgm tool, for example – to see when global checkpoints are written:

ndb_mgm> all clusterlog checkpoint=15
Executing CLUSTERLOG CHECKPOINT=15 on node 3 OK!
Executing CLUSTERLOG CHECKPOINT=15 on node 4 OK!

MySQL Cluster presentation at Oracle Open World 2010

As part of “MySQL Sunday” at this year’s Oracle Open World, Mat Keep and I will be presenting on the latest MySQL Cluster features. We’ll be presenting at 15:30 (Pacific Time) on 19th September (the event starts with a key note at 12:30).

If you’re attending Oracle Open World then please indicate that you’d like to attend the MySQL Sunday when you register. If you aren’t planning to go to Oracle Open World but will be in the San Francisco area then buying a Discover pass (only $50 if you register by 16 July) will get you into the MySQL Sunday sessions. Register here.

For details on the presentations and speakers, check here.

Webinar today – Scaling Web Services with MySQL Cluster, Part 1: An Alternative to MySQL Server & memcached

The replay of this webinar is now available from http://www.mysql.com/news-and-events/on-demand-webinars/display-od-545.html

MySQL and memcached has become, and will remain, the foundation for many dynamic web services with proven deployments in some of the largest and most prolific names on the web. There are classes of web services however that are update-intensive, demanding real-time responsiveness and continuous availability. In these cases, MySQL Cluster provides the familiarity and ease-of-use of the regular MySQL Server, while delivering significantly higher levels of write performance with less complexity, lower latency and 99.999% availability. This webinar will discuss the use-cases for both approaches, and provide an insight into how MySQL Cluster is enabling users to scale their update-intensive web services.

The webinar starts at 09:00 Pacific/17:00 UK/18:00 CET today (June 9th 2010).

Still time to register (for free) at http://www.mysql.com/news-and-events/web-seminars/display-545.html – even if you can’t attend, this way you’ll get sent a link to the charts and replay.

EAVB_ZZDDGVLWJR

Download, install, configure, run and test MySQL Cluster in under 15 minutes

Single host Cluster

A series of quick-start guides are now available to get you up and running with MySQL Cluster in as little time as possible; they are available for LINUX/Mac OS X, Windows and Solaris. The configuration is intentionally a simple one – 2 data nodes, 1 management node and 1 MySQL Server. Once you have this up and running, your next experiment may be to extend this over multiple hosts.

Download the Quick Start Guide for your platform below:

• MySQL Cluster Quick Start for Linux – also works for Mac OS X
• MySQL Cluster Quick Start for Windows
• MySQL Cluster Quick Start for Solaris

These links are also available from the MySQL Cluster download page.

The intent is that these guides should be simple to follow even if you have never tried MySQL Cluster (or even MySQL) before. Any comments or suggested improvements would be appreciated.