2 weeks ago I travelled to Sacramento, CA, to physically install the servers and storage for this new Oracle 11g data warehouse. The servers are 4 of the HP Proliant DL-585G6. These servers are pretty incredible, using the new AMD Opteron 6-core CPUs, with a total of 24 cores per server. In addition, each core has 3MB of cache. Each server came bundled with 128GB of RAM, which, when pooled together will give us a massive half terabyte of cache for Oracle 11g RAC to use!
The network consists of a pair of Cisco Catalyst 4900M switches. They have 24 ports each of the copper CX-4 10 gigabit Ethernet adapters, and each server has a pair of dual port Intel 10GbE adapters, so we can have bonded connections to both the public IP network and the private Oracle RAC interconnect. With half a terabyte of total cache and a 10 gigabit interconnect, cache memory hits through Oracle Cache Fusion should be extremely fast, almost as fast as local memory.
On the SAN network we have a pair of HP/Brocade 8/24 8 gigabit switches, with 4 of the 8gb Qlogic adapters per server, for a total SAN bandwidth of 32gb per server. Each storage array has 4x 4gb connections to the SAN, for a total of 16gb per array.
The storage consists of 4 of the HP MSA 2000fc G2 storage arrays, each with an expansion HP MSA70 disk shelf, for a total of 49x 146GB 10k RPM disks (24 in the MSA 2000 itself, and 25 in the expansion shelf) per array, 196 disks total. This extra 25th disk might not leave you with an even number of disks, but it works great as a global hot spare in the event of a disk failure.
I spent the week installing and configuring the storage and servers, and getting everything on the network. Thanks to my network engineer, James, for helping on the Cisco side. At this point all of my HP/Brocade switches have been zoned, my servers have all had a base Red Hat Linux 5 installation completed, and I’m ready to start benchmarking storage. Now that everything is network accessible and physically cabled, I can head back home and do the rest remote.
Here’s what it looks like – Storage on the top, servers on the bottom, SAN switches at the very top, and you can’t see them, but network switches in the middle facing the back:
No comments:
Post a Comment