Thursday, March 11, 2010

Benchmarking multi-server Data Warehouse performance

I figured out a way to run a benchmark across multiple servers in the cluster simultaneously.  Basically, I have all 4 servers synchronized to an NTP time source, and I scheduled an at job to run the same benchmark at the same time on all 4 servers.  This way, the data points align in time.  Each data point consists of exactly 60 seconds of I/O traffic.  Watching all 4 nodes I tail –f the .csv file so I can watch each data point be reported in real time after it is captured, and each node is in sync with each other down to the split second.

Here are the results of the 2 node benchmark I ran, pushing just over 4GB/sec:

image

And the 4 node benchmark, getting close to 5GB/sec.  Almost 5 gigabytes a second is pretty impressive for a storage system that cost less than $200K…  I know I could build a server with multiple attached MSA70 shelves that would have faster performance, but this is a shared everything fully scale out architecture, not a single node “all eggs in one basket” scenario:

image

No comments:

Post a Comment