Optimizing Performance of the MySQL Cluster Database – White Paper update

Engineering threads within a Data Node

This paper steps you through:

• Identifying if your application is a good fit for MySQL Cluster
• Measuring performance and identifying problem performance areas to address
• Optimizing performance:
  • Access patterns
  • Using Adaptive Query Localization for complex Joins
  • Distribution aware applications
  • Batching operations
  • Schema optimizations
  • Query optimization
  • Parameter tuning
  • Connection pools
  • Multi-Threaded Data Nodes
  • Alternative APIs
  • Hardware enhancements
  • Miscellaneous
• Scaling MySQL Cluster by Adding Nodes

As well as the kind of regular updates that are needed from time to time, this version includes the extra opportunities for optimizations that are available with MySQL Cluster 7.2 such as faster joins and engineering the threads within a multi-threaded data node.

As a reminder, I’ll be covering much of this material in an upcoming webinar.

MySQL Cluster : Delivering Breakthrough Performance (upcoming webinar)

MySQL Cluster partitioning key

The replay of this webinar is now available from here.

Here’s the agenda (hoping that I can squeeze it all in!):

• Introduction to MySQL Cluster
• Where does MySQL Cluster fit?
• Benchmarks:
  • ANALYZE TABLE
  • Access patterns
  • AQL (fast JOINs)
  • Distribution aware
  • Batching
  • Schema optimisations
  • Connection pooling
  • Multi-threaded data nodes
  • High performance NoSQL APIs
  • Hardware choices
  • More tips
• The measure/optimise loop
• Techniques to boost performance
• Scaling out
• Other useful resources

The session starts at 9:00 am UK time / 10:00 am Central European time.

MySQL Cluster Database 7: Performance Benchmark

(Note that this is a copy of the original article from Hasham Pathan posted on 21st April 2009).

Summary of Results:

We recently conducted a Performance Benchmark of the newly released version of MySQL Cluster database software version 7.0 using the DBT2 Benchmark. The results are very impressive.

Highlight of the results:

• For 4 Node Cluster, we achieved 251,000 Transactions per minute which is more than 4X improvement over the MySQL Cluster 6.3 release.
• For 2 Node Cluster, we achieved 143,000 Transactions per minute which is more than 4X improvement over the MySQL Cluster 6.3 release.
• Whether a user is looking to take advantage of the latest multi-core, multi-thread server processors, or is seeking to reduce space and energy consumption of their Cluster hardware, MySQL Cluster 7.0 provides a higher throughput, more efficient solution than previous generations of the MySQL Cluster database.

Note that these results were obtained using 1GB Gigabit Ethernet. We expect improved performance for high speed cluster interconnect like InfiniBand and Dolphinics interconnect solutions. Testing using these interconnects is currently underway.

Some of the KEY features of MySQL cluster 7.0 include “ability to add nodes and node groups online” and “Data node multithreading support” You can look at the list of new feature available in MySQL cluster 7.0 here.

Deployment Architecture and Configuration Details:

The topology diagram for 2 Node Scenario

In the case of a 2 node scenario, the data node processes were running on TWO Sun Fire x4450 system with 8 processor cores per data node. The MySQL server nodes were running a combination of Sun Fire x4450 systems and Sun Fire x4600 systems as shown in the deployment diagram below.

The topology diagram for 4 Node Scenario

In the case of a 4 node scenario, FOUR x4450 system were used to deploy the Data Nodes, each data node using 8 cores. The MySQL Server nodes were running on a combination of TWO Sun Fire x4600, ONE Sun Fire x4240 and FOUR Sun Fire x4450 systems.(co-located with data node systems). Other relevant details are captured in the deployment topology diagram.

Detailed Results:

Four Data Node Results

Two Data Node Results

Important Notes:

1. Each MySQL server was bound to 3 processor cores using “processor set” feature(to accommodate more MySQL Servers on the available hardware). It is observed that we get increased TPM/MySQL Server and slight decrease in the response time with 4 processor cores per MySQL server). An example: With 3 processor cores assigned for the MySQL server process, we get the TPM of 12988 for 1 Server, 20 Warehouses. With 4 processor cores, the TPM increases to 19113 with decrease in the response time from 52ms to 34ms.
2. TPM – New-order transactions per minute. It should be noted that the each transaction comprises of about 30 queries, so average response time per query if calculated would be ~3ms. More details on the “transaction” are available in the TPC-C specification document and DBT2 code base.
3. The MySQL 6.3 Cluster saturate with fewer number of MySQL servers. For comparison purpose, we ran the tests with equal number of MySQL servers for both MySQL Cluster 6.3 and 7.0

Benchmark Description

Database Test 2 (DBT-2) : DBT2 is an open source benchmark developed by OSDL (Open Source Development Labs ). Significant updates to the benchmark were made to simplify its ability to run with a clustered database such as MySQL Cluster Carrier Grade Edition. DBT2 simulates a typical OLTP (Online Transaction Processing) application that performs transactions with around ten to twenty SQL statements per transaction within five distinct transaction types. The DBT2 benchmark can be used as a good indicator in regards to the expected performance of any application that performs fairly simple transactions and executes these transactions in a repetitive fashion.

For the benchmark results above, DBT2 and MySQL Cluster were configured as an “in memory” database to simulate typical “real-time” database configurations. It should be noted that performance results are being measured as new-order transactions per minute (TPM). The changes made to the DBT2 benchmark are documented and can be found on the SourceForge page for DBT2 downloads. It can also be downloaded from ww.iclaustron.com.

Additional Information:

More information on MySQL Cluster

• MySQL Cluster 7 Architecture and New Features
• MySQL Cluster Evaluation Guide

Glimpse of the future – massively improved JOIN performance for MySQL Cluster

MySQL Cluster Push-Down Joins

A great chance to find out about the “SPJ” project that’s under way to improve the performance of complex table JOINs. A free webinar has been scheduled for Thursday, November 04 at 09:00 Pacific time (16:00 UK; 17:00 Central European Time) – just register for the webinar at mysql.com. Even if you can’t attend, by registering you’ll get an email telling you where to watch the replay shortly after the webinar.

MySQL Cluster performance has always been extremely high and scalable when the work load is primarily primary key reads and write but complex JOINS (many tables in the JOIN and/or a large number of results from the first part of the query) have traditionally been much slower than when using other storage engines. Work is underway to address this and SPJ is the name we’ve been using.

Traditionally, JOINs have been performed as Nested Loop JOINs in the MySQL Server which is fine when all of the data is held there (e.g. MyISAM) but when the data is held externally (in Cluster’s case, in the data nodes) it can result in a massive amount of messaging. SPJ works by pushing the processing of JOINs down into the data nodes where they can be performed much more efficiently as the data is local.

As well as finding out about the implementation, you’ll also learn:

• What queries benefit -> how you might tweak you application to get the biggest benefits
• The kind of performance improvements you might expect to see
• How to try the (pre-GA!) software for yourself.

Note that (as always) Oracle reserves the right to alter the timing and/or existence of new product releases.

Upcoming webinar – MySQL performance tuning best practices

MySQL Cluster Performance Tuning Best Practices

Are you experiencing current performance bottlenecks in your high availability applications ? Are you designing a new mission-critical application and want to know how best to structure your schema and index strategy for optimal performance? Interested in how to transform your SQL into faster, more efficient queries?

Benefits of connection pooling

Then this free web presentation is for you! You will get expert insight and learn best practices to help you identify those areas of database and application design that will give you the greatest benefits for performance when using MySQL Cluster.

We will discuss guidelines and best practices covering the following areas:

• General Design Concepts and Guidelines
• Schema Optimization
  • BLOB/Text vs VARBINARY/VARCHAR
  • Partition by Key
• Index Selection and Tuning
• Basic Query Tuning
• MySQL Cluster Parameter Tuning Guidelines
• Tools to accelerate application development and testing

Johan Andersson and Mat Keep will be presenting and I’ll be helping with the Q&A.

Tuesday, February 2, 2009: 10:00 CET (Central European Time)/ 09:00 UK/GMT.

Register here at mysql.com.

Free MySQL Cluster Performance Tuning webinar – TODAY!

MySQL Cluster Performance Tuning Best Practices

Are you experiencing current performance bottlenecks in your high availability applications ? Are you designing a new mission-critical application and want to know how best to structure your schema and index strategy for optimal performance? Interested in how to transform your SQL into faster, more efficient queries?

Then this free web presentation is for you! You will get expert insight and learn best practices to help you identify those areas of database and application design that will give you the greatest benefits for performance when using MySQL Cluster.

We will discuss guidelines and best practices covering the following areas:

• General Design Concepts and Guidelines
• Schema Optimization
  • BLOB/Text vs VARBINARY/VARCHAR
  • Partition by Key
• Index Selection and Tuning
• Basic Query Tuning
• MySQL Cluster Parameter Tuning Guidelines
• Tools to accelerate application development and testing

Johan Andersson and Mat Keep will be presenting and myself and Alex Yu handling the Q&A.

Tuesday, November 03, 2009: 10:00 Pacific time (America)

Tue, Nov 03: 08:00 Hawaii time

Tue, Nov 03: 11:00 Mountain time (America)

Tue, Nov 03: 12:00 Central time (America)

Tue, Nov 03: 13:00 Eastern time (America)

Tue, Nov 03: 18:00 UTC

Tue, Nov 03: 18:00 Western European time

Tue, Nov 03: 19:00 Central European time

Tue, Nov 03: 20:00 Eastern European time

Register here.