NetApp E5460 storage system
720 x 3TB SAS drives
SAS, NFS, gPFS
IBM Blue Gene/P
IBM Power 775
Red Hat Enterprise Linux®
ICMThe Interdisciplinary Centre for Mathematical and Computational Modelling (ICM; www.icm.edu.pl) was founded in 1993 at the University of Warsaw, Poland. ICM’s mission is to support a wide range of research areas, the success of which depends on the development of computational science. This includes biomedical sciences, physics, chemistry and theoretical biology, materials technologies, earth sciences, astronomy, and modeling of nonlinear processes in multiscale systems. ICM provides both a high-performance computational infrastructure and support for the Polish research community, being also part of the European Grid Infrastructure and the Partnership for Advanced Computing in Europe (PRACE) initiative.
Founded in 1816, the University of Warsaw (www.uw.edu.pl/en/) educates over 53,500 students, employs over 6,300 people, and hosts 20 faculties and 30 other units. It is listed among the top 4% of the world-class universities.
Optimize data storage
ICM is one of Poland’s leading high-performance computing (HPC) centers. Together with the HPC centers in Cracow and Poznan, ICM plays a major role in a national project called “HPC Infrastructure for Grand Challenges of Science and Engineering” (POWIEW). Cofounded by the European Union and under the patronage of the Polish Ministry of Science and Higher Education, the project kicked off in 2011. It includes the installation and operation of a leading-edge computing infrastructure across three cities, open to all researchers and real-world science and engineering applications.
Now in its 20th year, ICM operates an environment with compute and storage resources from six leading ICT vendors, including NetApp and IBM. IBM Blue Gene/P and IBM Power 775 systems serve ICM’s part of the POWIEW infrastructure. When the associated storage ran low on capacity, the team decided to deploy a central storage system for both temporary data from the HPC environment and long-term data such as home directories.
“We needed an efficient high-end solution with at least 6GB/sec for sequential data throughput, scalable capacity, and high density because data center space is scarce,” says Arkadiusz Niegowski, senior system engineer at ICM, who also defined the requirements. “As usual with governmental organizations, we issued a public tender. Thus, the best price combined with high density would determine the winning solution.”
Best value for money:
The tender resulted in five offers with three different solutions and twice a NetApp E-Series system. The systems are architected for performance, density, and modular flexibility to meet a wide range of data-intensive workloads; 99.999% availability, redundant components, automated path failover, and online administration help organizations stay productive 24/7.
NetApp Poland and its partner ADT Group won the project due to maximum storage density. The proposed NetApp E5460 houses 2.1PB of raw capacity on 720 SAS disks in just 12 shelves. Thus, the storage occupies a footprint of only 1.2 square meters in the data center.
The implementation was flawless. Within a few days, ADT Group had set up and tuned the system. After passing all performance tests, the NetApp E5460 was signed off as planned. Now, it was up to ICM to integrate the E5460 fully with the company’s multivendor environment and optimize user access. “Deploy cabling efficiently, design file system partitions for NFS and gPFS, mount user groups and servers, define data export paths—we had a plenty of things to do,” says Dr. Maciej Filocha, project manager and HPC coordinator at ICM. “It’s a demanding environment, but the flexibility of the E5460 is appealing.”
High performance, maximum protection
NetApp E5460 storage meets high performance and capacity requirements without sacrificing simplicity and efficiency. ICM can easily monitor and manage the hardware with NetApp SANtricity® software. The team chose a RAID 6 configuration, which protects against two simultaneous disk drive failures. It was even possible to increase the data throughput to 9GB/sec for write and 12GB/sec for read access.
Niegowski is very satisfied with the solution: “The storage is just running fine. It’s sufficient to check the system’s health once a week. In the nine months of its productive operation, there was only one failed disk drive. Given the extensive use of the storage and compared to other systems, this is really good.” In addition, system rebuilds are up to eight times faster than with a traditional RAID, thanks to NetApp Dynamic Disk Pools technology.
Set for data growth
The storage is sized to easily meet changing capacity needs for the temporary data resulting from a variety of computation applications. However, ICM expects higher growth rates in a couple of months because the complete POWIEW infrastructure is now close to maturity, with all equipment up and running. This challenge can be addressed easily with the addition of more disk shelves
Serve science, commerce,
and the public with big data
Performance, availability, and reliability count most in HPC environments. Some processes run for a complete month, and file sizes vary from 20GB to 70TB and even more. ICM’s infrastructure services are very well perceived and sought after. The team must carefully schedule POWIEW and other HPC resources for the research community.
If the storage fails, ICM can bypass one or two hours without getting behind schedule. However, the reliability of the storage helps to stay on track and also delivers very popular services such as the numerical weather forecast on time. Every six hours ICM publishes new forecast data for the atmosphere over Central Europe and the Baltic Sea. The service is free of charge and is widely acknowledged for nautical, sportive, and business purposes. Several wind farms in Poland use ICM data for day-to-day forecasting of renewable energy production.
“Our infrastructure must perform as it should to deliver trusted services to our community. In fact, this is a matter of reputation, and the reliability of the E5460 adds to it. Instead of spending time on operational tasks, we can support users to make the most of our resources from entrylevel to high-end HPC. This is what counts and helps us deliver on ICM’s mission,” summarizes Dr. Filocha.
High performance computing
Optimize data storage capabilities in a highly heterogeneous HPC environment. The solution should meet growth, performance, and space efficiency requirements.
Deploy a high-performance, highdensity NetApp® E5460 solution to accommodate short-term and long-term data from a range of computation applications.
- Outperformed performance requirement of at least 6GB/sec
- Achieved a data center footprint of only 39U for 2.1PB of storage
- Delivered on-time compute and storage services to computation applications, including the regularly updated weather forecast
- Added to ICM’s reputation as a leading HPC center