Doxygen output for MySQL Cluster NDB API & MGM API

NDB API Documentation

Creating a simple Cluster on a single LINUX host

It isn’t necessarily immediately obvious how to set up a Cluster on LINUX; this post attempts to show how to get a simple Cluster up and running. For simplicity, all of the nodes will run on a single host – a subsequent post will take the subsequent steps of moving some of them to a second host. As with my Windows post the Cluster will contain the following nodes:

• 1 Management node (ndb_mgmd)
• 2 Data nodes (ndbd)
• 3 MySQL Server (API) nodes (mysqld)

Downloading and installing

Browse to the MySQL Cluster LINUX download page at mysql.com and download the correct version (32 or 64 bit) and store it in the desired directory (in my case, /home/billy/mysql) and then extract and rename the new folder to something easier to work with…

[billy@ws1 mysql]$ tar xvf mysql-cluster-gpl-7.0.6-linux-x86_64-glibc23.tar.gz
[billy@ws1 mysql]$ mv mysql-cluster-gpl-7.0.6-linux-x86_64-glibc23 7_0_6

Create 3 data folders (one for each of the MySQL API – mysqld – processes) and setup the files that will be needed for them to run correctly…

[billy@ws1 mysql]$ cd 7_0_6/data
[billy@ws1 data]$ mkdir data1 data2 data3
[billy@ws1 data]$ mkdir data1/mysql data1/test data2/mysql data2/test data3/mysql data3/test
[billy@ws1 7_0_6]$ cd ..
[billy@ws1 7_0_6]$ scripts/mysql_install_db --basedir=/home/billy/mysql/7_0_6 --datadir=/home/billy/mysql/7_0_6/data/data1
[billy@ws1 7_0_6]$ scripts/mysql_install_db --basedir=/home/billy/mysql/7_0_6 --datadir=/home/billy/mysql/7_0_6/data/data2
[billy@ws1 7_0_6]$ scripts/mysql_install_db --basedir=/home/billy/mysql/7_0_6 --datadir=/home/billy/mysql/7_0_6/data/data3

Configure and run the Cluster

Create a sub-directory called “conf” and create the following 4 files there:

config.ini

[ndbd default]
noofreplicas=2

[ndbd]
hostname=localhost
id=2

[ndbd]
hostname=localhost
id=3

[ndb_mgmd]
id = 1
hostname=localhost

[mysqld]
id=4
hostname=localhost

[mysqld]
id=5
hostname=localhost

[mysqld]
id=6
hostname=localhost

my.1.conf

[mysqld]
ndb-nodeid=4
ndbcluster
datadir=/home/billy/mysql/7_0_6/data/data1
basedir=/home/billy/mysql/7_0_6
port=3306
server-id=1
log-bin

my.2.conf

[mysqld]
ndb-nodeid=5
ndbcluster
datadir=/home/billy/mysql/7_0_6/data/data2
basedir=/home/billy/mysql/7_0_6
port=3307
server-id=2
log-bin

my.3.conf

[mysqld]
ndb-nodeid=6
ndbcluster
datadir=/home/billy/mysql/7_0_6/data/data3
basedir=/home/billy/mysql/7_0_6
port=3308
server-id=3
log-bin

Those files configure the nodes that make up the Cluster. From a command prompt window, launch the management node:

[billy@ws1 7_0_6]$ bin/ndb_mgmd --initial -f conf/config.ini --configdir=/home/billy/mysql/7_0_6/conf
2009-06-17 13:00:08 [MgmSrvr] INFO     -- NDB Cluster Management Server. mysql-5.1.34 ndb-7.0.6
2009-06-17 13:00:08 [MgmSrvr] INFO     -- Reading cluster configuration from 'conf/config.ini'

Check that the management node is up and running:

[billy@ws1 7_0_6]$ bin/ndb_mgm
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]    2 node(s)
id=2 (not connected, accepting connect from localhost)
id=3 (not connected, accepting connect from localhost)

[ndb_mgmd(MGM)]    1 node(s)
id=1    @localhost  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)]    3 node(s)
id=4 (not connected, accepting connect from localhost)
id=5 (not connected, accepting connect from localhost)
id=6 (not connected, accepting connect from localhost)
ndb_mgm> quit

and then start the 2 data nodes (ndbd) and 3 MySQL API/Server nodes (ndbd) and then check that they’re all up and running:

[billy@ws1 7_0_6]$ bin/ndbd --initial -c localhost:1186
2009-06-17 13:05:47 [ndbd] INFO     -- Configuration fetched from 'localhost:1186', generation: 1
[billy@ws1 7_0_6]$ bin/ndbd --initial -c localhost:1186
2009-06-17 13:05:51 [ndbd] INFO     -- Configuration fetched from 'localhost:1186', generation: 1

[billy@ws1 7_0_6]$ bin/mysqld --defaults-file=conf/my.1.conf&
[billy@ws1 7_0_6]$ bin/mysqld --defaults-file=conf/my.2.conf&
[billy@ws1 7_0_6]$ bin/mysqld --defaults-file=conf/my.3.conf&

[billy@ws1 7_0_6]$ bin/ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]    2 node(s)
id=2    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0, Master)
id=3    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0)

[ndb_mgmd(MGM)]    1 node(s)
id=1    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)]    3 node(s)
id=4    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)
id=5    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)
id=6    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)
ndb_mgm> quit

Using the Cluster

There are now 3 API nodes/MySQL Servers/mysqlds running; all accessing the same data. Each of those nodes can be accessed by the mysql client using the ports that were configured in the my.X.cnf files. For example, we can access the first of those nodes (node 4) in the following way (each API node is accessed using the port number in its associate my.X.cnf file:

[billy@ws1 7_0_6]$ bin/mysql -h localhost -P 3306
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.1.34-ndb-7.0.6-cluster-gpl-log MySQL Cluster Server (GPL)
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use test;
Database changed
mysql> create table assets (name varchar(30) not null primary key,
-> value int) engine=ndb;
090617 13:21:36 [Note] NDB Binlog: CREATE TABLE Event: REPL$test/assets
090617 13:21:36 [Note] NDB Binlog: logging ./test/assets (UPDATED,USE_WRITE)
090617 13:21:37 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$test/assets
090617 13:21:37 [Note] NDB Binlog: DISCOVER TABLE Event: REPL$test/assets
090617 13:21:37 [Note] NDB Binlog: logging ./test/assets (UPDATED,USE_WRITE)
090617 13:21:37 [Note] NDB Binlog: logging ./test/assets (UPDATED,USE_WRITE)
Query OK, 0 rows affected (0.99 sec)
mysql> insert into assets values ('Car','1900');
Query OK, 1 row affected (0.03 sec)
mysql> select * from assets;
+------+-------+
| name | value |
+------+-------+
| Car  |  1900 |
+------+-------+
1 row in set (0.00 sec)
mysql> quit
Bye

Note that as this table is using the ndb (MySQL Cluster) storage engine, the data is actually held in the data nodes rather than in the SQL node and so we can access the exact same data from either of the other 2 SQL nodes:

[billy@ws1 7_0_6]$ bin/mysql -h localhost -P 3307
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.1.34-ndb-7.0.6-cluster-gpl-log MySQL Cluster Server (GPL)
type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from assets;
+------+-------+
| name | value |
+------+-------+
| Car  |  1900 |
+------+-------+
1 row in set (0.00 sec)
mysql> quit
Bye

Your next steps

This is a very simple, contrived set up – in any sensible deployment, the nodes would be spread across multiple physical hosts in the interests of performance and redundancy (take a look at the new article (Deploying MySQL Cluster over multiple host) to see how to do that). You’d also set several more variables in the configuration files in order to size and tune your Cluster.

My first Cluster running on Windows

I figured that it was time to check out how to install, configure, run and use MySQL Cluster on Windows. To keep things simple, this first Cluster will all run on a single host but includes these nodes:

• 1 Management node (ndb_mgmd)
• 2 Data nodes (ndbd)
• 3 MySQL Server (API) nodes (mysqld)

Downloading and installing

Browse to the Windows section of the MySQL Cluster 7.0 download page and download the installer (32 or 64 bit).

MySQL Cluster Windows Installer

Run the .msi file and choose the “Custom” option. Don’t worry about the fact that it’s branded as “MySQL Server 7.0″ and that you’ll go on to see adverts for MySQL Enterprise – that’s just an artefact of how the installer was put together.

On the next screen, I decided to change the “Install to” directory to “c:\mysql” – not essential but it saves some typing later.

Go ahead and install the software and then you’ll be asked if you want to configure the server – uncheck that as we’ll want to tailor the configuration so that it works with our Cluster.

There are a couple of changes you need to make to your Windows configuration before going any further:

1. Add the new bin folder to your path (in my case “C:\mysql\bin”)
2. Make hidden files visible (needed in order to set up multiple MySQL Server processes on the same machine)

Configure and run the Cluster

Copy the contents of “C:\ProgramData\MySQL\MySQL Server 7.0\data” to “C:\ProgramData\MySQL\MySQL Server 7.0\data4″, “C:\ProgramData\MySQL\MySQL Server 7.0\data5″ and “C:\ProgramData\MySQL\MySQL Server 7.0\data6″. Note that this assumes that you’ve already made hidden files visible. Each of these folders will be used by one of the mysqld processes.

Create the folder “c:\mysql\cluster” and then create the following files there:

config.ini

[ndbd default]
noofreplicas=2

[ndbd]
hostname=localhost
id=2

[ndbd]
hostname=localhost
id=3

[ndb_mgmd]
id = 1
hostname=localhost

[mysqld]
id=4
hostname=localhost

[mysqld]
id=5
hostname=localhost

[mysqld]
id=6
hostname=localhost

my.4.cnf

[mysqld]
ndb-nodeid=4
ndbcluster
datadir="C:\ProgramData\MySQL\MySQL Server 7.0\data4"
port=3306
server-id=3306

my.5.cnf

[mysqld]
ndb-nodeid=5
ndbcluster
datadir="C:\ProgramData\MySQL\MySQL Server 7.0\data5"
port=3307
server-id=3307

my.6.cnf

[mysqld]
ndb-nodeid=6
ndbcluster
datadir="C:\ProgramData\MySQL\MySQL Server 7.0\data6"
port=3308
server-id=3308

Those files configure the nodes that make up the Cluster.

From a command prompt window, launch the management node:

C:\Users\Andrew>cd \mysql\cluster
C:\mysql\cluster>ndb_mgmd -f config.ini
2009-06-16 20:01:20 [MgmSrvr] INFO     -- NDB Cluster Management Server. mysql-5.1.34 ndb-7.0.6
2009-06-16 20:01:20 [MgmSrvr] INFO     -- The default config directory 'c:\mysql\mysql-cluster' does not exist. Trying to create it...
2009-06-16 20:01:20 [MgmSrvr] INFO     -- Sucessfully created config directory
2009-06-16 20:01:20 [MgmSrvr] INFO     -- Reading cluster configuration from 'config.ini'

and then from another window, check that the cluster has been defined:

 C:\Users\Andrew>ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm> show
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2 (not connected, accepting connect from localhost)
id=3 (not connected, accepting connect from localhost)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @localhost  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)]   3 node(s)
id=4 (not connected, accepting connect from localhost)
id=5 (not connected, accepting connect from localhost)
id=6 (not connected, accepting connect from localhost)

Fire up 2 more command prompt windows and launch the 2 data nodes:

C:\Users\Andrew>ndbd
2009-06-16 20:08:57 [ndbd] INFO     -- Configuration fetched from 'localhost:118
6', generation: 1
2009-06-16 20:08:57 [ndbd] INFO     -- Ndb started
NDBMT: non-mt
2009-06-16 20:08:57 [ndbd] INFO     -- NDB Cluster -- DB node 2
2009-06-16 20:08:57 [ndbd] INFO     -- mysql-5.1.34 ndb-7.0.6 --
2009-06-16 20:08:57 [ndbd] INFO     -- Ndbd_mem_manager::init(1) min: 84Mb initi
al: 104Mb
Adding 104Mb to ZONE_LO (1,3327)
2009-06-16 20:08:57 [ndbd] INFO     -- Start initiated (mysql-5.1.34 ndb-7.0.6)
WOPool::init(61, 9)
RWPool::init(22, 13)
RWPool::init(42, 18)
RWPool::init(62, 13)
Using 1 fragments per node
RWPool::init(c2, 18)
RWPool::init(e2, 14)
WOPool::init(41, 8 )
RWPool::init(82, 12)
RWPool::init(a2, 52)
WOPool::init(21, 5)

(repeat from another new window for the second data node).

After both data nodes (ndbd) have been launched, you should be able to see them through the management client:

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0, Master)
id=3    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)]   3 node(s)
id=4 (not connected, accepting connect from localhost)
id=5 (not connected, accepting connect from localhost)
id=6 (not connected, accepting connect from localhost)

Finally, the 3 MySQL Server/API nodes should be lauched from 3 new windows:

C:\Users\Andrew>cd \mysql\cluster
C:\mysql\cluster>mysqld --defaults-file=my.4.cnf

C:\Users\Andrew>cd \mysql\cluster
C:\mysql\cluster>mysqld --defaults-file=my.5.cnf

C:\Users\Andrew>cd \mysql\cluster
C:\mysql\cluster>mysqld --defaults-file=my.6.cnf

Now, just check that all of the Cluster nodes are now up and running from the management client…

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0, Master)
id=3    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)

[mysqld(API)]   3 node(s)
id=4    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)
id=5    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)
id=6    @127.0.0.1  (mysql-5.1.34 ndb-7.0.6)

Using the Cluster

There are now 3 API nodes/MySQL Servers/mgmds running; all accessing the same data. Each of those nodes can be accessed by the mysql client using the ports that were configured in the my.X.cnf files. For example, we can access the first of those nodes (node 4) in the following way from (yet another) window:

C:\Users\Andrew>mysql -h localhost -P 3306
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.34-ndb-7.0.6-cluster-gpl MySQL Cluster Server (GPLType 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use test;
Database changed
mysql> create table assets (name varchar(30) not null primary key, value int) engine=ndb;
Query OK, 0 rows affected (1.44 sec
mysql> insert into assets values ('car', 950);
Query OK, 1 row affected (0.00 sec
mysql> select * from assets;
+------+-------+
| name | value |
+------+-------+
| car  |   950 |
+------+-------+
1 row in set (0.00 sec
mysql> insert into assets2 values ('car', 950);
Query OK, 1 row affected (0.00 sec)

To check that everything is working correctly, we can access the same database through another of the API nodes:

C:\Users\Andrew>mysql -h localhost -P 3307
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.34-ndb-7.0.6-cluster-gpl MySQL Cluster Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use test;
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| assets         |
+----------------+
1 row in set (0.06 sec)

mysql> select * from assets;
+------+-------+
| name | value |
+------+-------+
| car  |   950 |
+------+-------+
1 row in set (0.09 sec)

It’s important to note that the table (and its contents) of any table created using the ndb storage engine can be accessed through any of the API nodes but those created using other storage engines are local to each of the API nodes (MySQL Servers).

Your next steps

This is a very simple, contrived set up – in any sensible deployment, the nodes would be spread accross multiple physical hosts in the interests of performance and redundancy. You’d also set several more variables in the configuration files in order to size and tune your Cluster. Finally, you’d likely want to have some of these processes running as daemons or services rather than firing up so many windows.

It’s important to note that Windows is not a fully supported platform for MySQL Cluster. If you have an interest in deploying a production system on Windows then please contact me at andrew@clusterdb.com

New MySQL Cluster maintenance release: 7.0.6

MySQL Cluster version 7.0.6 has been released. MySQL Cluster 7.0.6 is available in source and binary form for a number of platforms from our download pages.

The release in source form can in addition be downloaded from the MySQL FTP server.

MySQL Cluster version 7.0.6 is using MySQL Server version 5.1.34 and contains all bug fixes and changes that MySQL Server 5.1.33 and 5.1.34 contain.
Following changes were made in MySQL Cluster 7.0.6 since the previous release 7.0.5:

Functionality added or changed:

• Important Note: It is possible in MySQL Cluster NDB 7.0.6 and later   to perform online upgrades from MySQL Cluster NDB 6.3.8 and later MySQL   Cluster NDB 6.3 releases, or from MySQL Cluster NDB 7.0.5 or later MySQL Cluster NDB 7.0 releases.
• It was not possible to perform an online upgrade from any MySQL Cluster  NDB 6.x release to MySQL Cluster NDB 7.0.5 or any to earlier MySQL Cluster NDB 7.0 release. (Bug#44294)
• The ndb_config utility program can now provide an offline dump of all MySQL Cluster configuration parameters including information such as default and permitted values, brief description, and applicable section of the config.ini file.

The documentation has been updated accordingly and a full change log will be found.

For more information about changes and bugs fixed in MySQL Server 5.1.33 and 5.1.34 please refer to the reference manual.

Webinar replay available: What’s New in the Next Generation of MySQL Cluster?

There is another live webinar on MySQL Cluster 7.0 scheduled for 27th May 09:00 (Western European time) / 08:00 (UK time) / 11:00 (Eastern European time) and you can register here.

Alternatively, if you can’t wait that long then you can watch/listen to the play-back of the original webinar.

Note that you may need to install the WebEx Player (Windows and Mac) – unfortunately, there doesn’t seem to be a simple, supported solution for LINUX users but you can always download the slides (PDF format) from the same link or else wait until 27th May and view it live.

What’s New in the Next Generation of MySQL Cluster?
Thursday, April 30, 2009

Whether you’re racing to introduce a new service, or trying to manage an avalanche of data in real time, your database has to be scalable, fast and highly available to meet ever-changing market conditions and stringent SLAs.

By attending this webinar, you will learn more about MySQL Cluster 7, and how it enables you to deliver 99.999% database availability, with real time performance and linear scalability, while slashing TCO.

Boasting a range of breakthrough capabilities, the MySQL Cluster 7 will enable you to stay ahead of your most demanding, mission-critical application requirements. Enhanced scalability delivers higher database performance with fewer nodes to simplify deployment and administration. Expanded platform support and interoperability delivers more choice in both the development and deployment of MySQL Cluster. Simplified cluster monitoring tools reduce DBA administration overhead and operational costs.

Presented by:

• Andrew Morgan, MySQL Product Management
• Matthew Keep, MySQL Product Management

Live Webinar: What’s New in the Next Generation of MySQL Cluster?

Following on from the announcement  at the User Conference last week that MySQL Cluster 7.0 is now generally available, Mat Keep and I will present the first public webcast this Thursday. If you’d like to attend but the time isn’t convenient then you can either watch/listen to the recording after the event or wait for the next one to be announced (will be aimed at different time-zones).

Details…..

What’s New in the Next Generation of MySQL Cluster?
Thursday, April 30, 2009

Whether you’re racing to introduce a new service, or trying to manage an avalanche of data in real time, your database has to be scalable, fast and highly available to meet ever-changing market conditions and stringent SLAs.

By attending this webinar, you will learn more about MySQL Cluster 7, and how it enables you to deliver 99.999% database availability, with real time performance and linear scalability, while slashing TCO.

Boasting a range of breakthrough capabilities, the MySQL Cluster 7 will enable you to stay ahead of your most demanding, mission-critical application requirements. Enhanced scalability delivers higher database performance with fewer nodes to simplify deployment and administration. Expanded platform support and interoperability delivers more choice in both the development and deployment of MySQL Cluster. Simplified cluster monitoring tools reduce DBA administration overhead and operational costs.
WHO:

* Andrew Morgan, MySQL Product Management
* Matthew Keep, MySQL Product Management

WHAT: What’s New in the Next Generation of MySQL Cluster? web presentation.

WHEN:
Thursday, April 30, 2009: 10:00 Pacific time (America)
Thu, Apr 30:      07:00 Hawaii time
Thu, Apr 30:      11:00 Mountain time (America)
Thu, Apr 30:      12:00 Central time (America)
Thu, Apr 30:      13:00 Eastern time (America)
Thu, Apr 30:      17:00 UTC
Thu, Apr 30:      18:00 Western European time
Thu, Apr 30:      19:00 Central European time
Thu, Apr 30:      20:00 Eastern European time

The presentation will be approximately 45 minutes long followed by Q&A.
WHERE: Simply access the web seminar from the comfort of your own office.

Register for the live webinar here:
http://www.mysql.com/news-and-events/web-seminars/display-320.html

Download the MySQL Cluster Architecture and New Features whitepaper:
http://www.mysql.com/why-mysql/white-papers/mysql_wp_cluster7_architecture.php

MySQL Cluster 7.0 Launched

MySQL (part of Sun Microsystems) have just announced the release of MySQL Cluster 7.0. This is a major new version of the database.

The highlights of the release (as well as an overview of the MySQL Cluster architecture) have been published in a new white paper: MySQL Cluster 7.0: Architecture and New Features.

For those in a rush, here are the highlights of the highlights:

• Multi-threaded data nodes. Could previously exploit up to 2 cores/CPUs/threads for a single data node. This is extended to 8 cores by introducing a multi-threaded version of the ndb process. This delivers a very significant performance improvement if running on a host with more than  a dual core.
• On-line add node. The ability to add a new node group to an in-service Cluster without impacting service. Includes the ability to repartition the data in existing tables after the new node group has been added.

On-Line Add Node

• Carrier-Grade directory back end. Support for using MySQL Cluster as the data store for a directory server. Allows LDAP access to your data. OpenLDAP will be the first directory to release a driver for this.
• Multi-Threaded Disk Data File Access. Improves the performance for disk-based table data.
• Improved large record handling. Performance/network bandwidth improvements.
• Alpha version of a Windows port. Intended for development use only, not for deployment.
• Snapshot option for backups. Option to have the backup represent the state of the database at the point that the backup was started rather than when it finished. This lets you synchronise it with other, external events such as the  backing up of other databases.
• Caching of configuration data. When a node restarts, it will only adopt any changes to the config files if explicitly told to – avoiding the situation where one node restarts and ends up with different config data to the others.
• Transactions for schema changes. Safer way to update the schema that can tolerate node failures mid-operation.