HPC in Weather Forecast: Moving to the Cloud
Pages 14 - 31
Abstract
Many traditional High-Performance Computing systems rely on hardware performance as well as compiler parallelization. The increased demand on computing resources rapidly escalates costs, which are the bigger limitation nowadays in both public and private sector. Cutting costs stimulates the research for new solutions, which are being opened by Cloud Computing. Virtualization becomes the key to migration and delivering intensive computing applications becomes more proficient than ever, thus providing the motivation for a change to the Cloud. This paper presents a case study of how to migrate an application and optimize performance without significant cost growth based on Cloud Computing applied to weather forecasting.
References
[1]
AkhaniJ.ChuadharyS.SomaniG. 2011. Negotiation for resource allocation in IaaS cloud. Proceedings of the Fourth Annual ACM Bangalore Conference. Bangalore, India: ACM. 10.1145/1980422.1980437
[2]
Amazon Web Services LLC. n.d. AWS | Amazon Elastic Compute Cloud EC2 - Scalable Cloud Hosting. Retrieved from http://aws.amazon.com/ec2
[3]
Barr, J. 2011. Next Generation Cluster Computing on Amazon EC2 - The CC2 Instance Type. Retrieved from http://aws.typepad.com/aws/2011/11/next-generation-cluster-computing-on-amazon-ec2-the-cc2-instance-type.html
[4]
Beckett, J., & Bradfield, R. 2011. Power Efficiency Comparison of Enterprise-Class Blade Servers and Enclosures p. 48., Retrieved from www.dell.com/downloads/global/products/pedge/en/bladepowerstudywhitepaper_08112010_final.pdf.
[5]
Bellard, F. 2013. Open source processor emulator. Retrieved from http://wiki.qemu.org
[6]
Brown, J. R., & Dinu, V. 2013. High performance computing methods for the integration and analysis of biomedical data using {SAS}. Computer Methods and Programs in Biomedicine, 1123, 553-562. 24051021.
[7]
BunchC.ChohanN.KrintzC.ShamsK. 2011. Neptune: a domain specific language for deploying hpc software on cloud platforms. Proceedings of the 2nd International Workshop on Scientific Cloud Computing. San Jose, California, USA: ACM. 10.1145/1996109.1996120
[8]
BunchC.KrintzC. 2011. Enabling automated HPC / database deployment via the appscale hybrid cloud platform. Proceedings of the First Annual Workshop on High Performance Computing Meets Databases. Seattle, Washington, USA: ACM. 10.1145/2125636.2125642
[9]
C12G. 2013. The Open Source Toolkit for Data Center Virtualization. Retrieved from http://opennebula.org/
[10]
Carlyle, A. G., Harrell, S. L., & Smith, P. M. 2010. Cost-Effective HPC: The Community or the Cloud? In Cloud Computing Technology and Science CloudCom, 2010 IEEE Second International Conference on pp. 169-176.
[11]
Carvalho, D., Rocha, A., Gómez-Gesteira, M., & Santos, C. 2012. A sensitivity study of the {WRF} model in wind simulation for an area of high wind energy. Environmental Modelling & Software, 330, 23-34.
[12]
DudhiaJ.GillD. O.HendersonT. B.KlempJ. B.SkamarockW.WangW. 2011. The Weather Research and Forecast Model: software architecture and performance. In 11th Workshop on the Use of High Performance Computing in Meteorology p. 6. Reading, United Kingdom: European Centre for Medium Range Weather Forecasts.
[13]
Expósito, R. R., Taboada, G. L., Ramos, S., Touriño, J., & Doallo, R. 2013. Performance analysis of HPC applications in the cloud. Future Generation Computer Systems, 291, 218-229.
[14]
Friedley, A., & Lumsdaine, A. Communication Optimization Beyond MPI., 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum 2018-2021 2011. IEEE.
[15]
GhoshalD.CanonR. S.RamakrishnanL. 2011. I/O performance of virtualized cloud environments. Proceedings of the Second International Workshop on Data Intensive Computing in the Clouds. Seattle, Washington, USA: ACM. 10.1145/2087522.2087535
[16]
Gillam, L., & Li, B., O'Loughlin, J., & Tomar, A. P. 2013. Fair Benchmarking for Cloud Computing systems. Journal of Cloud Computing: Advances . Systems and Applications, 21, 6.
[17]
Glenis, V., McGough, A. S., Kutija, V., Kilsby, C., & Woodman, S. 2013. Flood modelling for cities using Cloud computing. Journal of Cloud Computing: Advances . Systems and Applications, 21, 7.
[18]
GMCUA. 2013. CliM@UA. Retrieved from http://climetua.fis.ua.pt
[19]
HeQ.ZhouS.KoblerB.DuffyD.McGlynnT. 2010. Case study for running HPC applications in public clouds. Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing. Chicago, Illinois: ACM. 10.1145/1851476.1851535
[20]
Hewlett-Packard Development Company. L. P. n.d. HP ProLiant BL460c Gen8 E5-2650v2 1P 32GB-R P220i/512 FBWC Server. Retrieved from http://www8.hp.com/pt/pt/products/proliant-servers/product-detail.html?oid=5409979#!tab=specs
[21]
Jackson, K. R., Ramakrishnan, L., Muriki, K., Canon, S., Cholia, S., & Shalf, J. ' Wright, N. J. 2010. Performance Analysis of High Performance Computing Applications on the Amazon Web Services Cloud. In Cloud Computing Technology and Science CloudCom, 2010 IEEE Second International Conference on pp. 159-168.
[22]
Jakovits, P., & Srirama, S. N. 2013. Adapting Scientific Applications to Cloud by Using Distributed Computing Frameworks. In Cluster, Cloud and Grid Computing CCGrid, 2013 13th IEEE/ACM International Symposium on pp. 164-167. 10.1109/CCGrid.2013.47
[23]
Jianhua, C., Qinming, H., Qinghua, G., & Dawei, H. 2008. Performance Measuring and Comparing of Virtual Machine Monitors. In Embedded and Ubiquitous Computing, 2008. EUC '08. IEEE/IFIP International Conference on Vol. 2, pp. 381-386.
[24]
Juve, G., Deelman, E., Berriman, G. B., Berman, B. P., & Maechling, P. 2012. An Evaluation of the Cost and Performance of Scientific Workflows on Amazon EC2. Journal of Grid Computing, 101, 5-21.
[25]
Lucas Simarro, J. L., Moreno-Vozmediano, R., Montero, R. S., & Llorente, I. M. 2011. Dynamic placement of virtual machines for cost optimization in multi-cloud environments. In High Performance Computing and Simulation HPCS, 2011 International Conference on pp. 1-7.
[26]
Michalakes, J., Loft, R., & Bourgeois, A. 2001. Performance-Portability and the Weather Research and Forecast Model. In HPC Asia. Queensland, Australia. Retrieved from http://www.mcs.anl.gov/~michalak/B/HPCAsia2001_michalakes.pdf
[27]
Montero, R. S., Moreno-Vozmediano, R., & Llorente, I. M. 2011. An elasticity model for High Throughput Computing clusters. Journal of Parallel and Distributed Computing, 716, 750-757.
[28]
NCEP. 2003. The GFS Atmospheric Model. G. C. and W. M. Branch, Ed.NCEP Office Note 442. Cam Springs Maryland: EMC.
[29]
NIST. 2011. Guide to Security for Full Virtualization Technologies p. 35. DIANE Publishing. Retrieved from http://csrc.nist.gov/publications/nistpubs/800-125/SP800-125-final.pdf
[30]
NowoczynskiP.SommerfieldJ.YanovichJ.ScottJ. R.ZhangZ.LevineM. 2012. The data supercell. Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the Campus and beyond. Chicago, Illinois: ACM. 10.1145/2335755.2335805
[31]
Red Hat, I. 2013. Kernel Based Virtual Machine. Retrieved from http://www.linux-kvm.org
[32]
Roberto, R. 2013. Analysis of I/O Performance on an Amazon EC2 Cluster Compute and High I/O Platform. Journal of Grid Computing, 114, 613-631.
[33]
Saini, S., Heistand, S., Jin, H., Chang, J., Hood, R., Mehrotra, P., & Biswas, R. 2012. An Application-based Performance Evaluation of NASA's Nebula Cloud Computing Platform. In High Performance Computing and Communication 2012 IEEE 9th International Conference on Embedded Software and Systems HPCC-ICESS, 2012 IEEE 14th International Conference on pp. 336-343.
[34]
Simons, J. E., & Buell, J. 2010. Virtualizing high performance computing. SIGOPS Oper. Syst. Rev., 444, 136-145.
[35]
Skamarock Klemp, J. 420 B., Dudhia, J., Gill, J., Barke, D. M., Wang, W. and Powers, J. G., W. C. 2005. A description of the Advance Reaserch WRF version. NCAR/TN-68+STR, Ed.NCAR Technical Note p. 88.
[36]
Sotomayor, B., Montero, R. S., Llorente, I. M., & Foster, I. 2009. Virtual infrastructure management in private and hybrid clouds. IEEE Internet Computing, 13, 14-22. IEEE Educational Activities Department.
[37]
Vecchiola, C., Pandey, S., & Buyya, R. 2009. High-Performance Cloud Computing: A View of Scientific Applications. In Pervasive Systems, Algorithms, and Networks ISPAN, 2009 10th International Symposium on pp. 4-16. 10.1109/I-SPAN.2009.150
[38]
Von Quast, H. M., Hauck, M., & Kounev, S. N. 2011. Evaluating and Modeling Virtualization Performance Overhead for Cloud Environments. CLOSER 2011. Retrieved from http://sdqweb.ipd.kit.edu/publications/descartes-pdfs/HuQuHaKo2011-CLOSER-ModelVirtOverhead.pdf
[39]
Voss, A., Barker, A., Asgari-Targhi, M., van Ballegooijen, A., & Sommerville, I. 2013. An elastic virtual infrastructure for research applications ELVIRA. Journal of Cloud Computing: Advances . Systems and Applications, 21, 20.
[40]
William, D. Norcott. 2013. IOzone Filesystem Benchmark. Retrieved from http://www.iozone.org
[41]
Zwiers, F. W., & von Storch, H. 2004. On the role of statistics in climate research. International Journal of Climatology, 246, 665-680.
Recommendations
Virtual InfiniBand clusters for HPC clouds
CloudCP '12: Proceedings of the 2nd International Workshop on Cloud Computing PlatformsHigh Performance Computing (HPC) employs fast interconnect technologies to provide low communication and synchronization latencies for tightly coupled parallel compute jobs. Contemporary HPC clusters have a fixed capacity and static runtime environments;...
Virtualized HPC: a contradiction in terms?
System virtualization has become the enabling technology to manage the increasing number of different applications inside data centers. The abstraction from the underlying hardware and the provision of multiple virtual machines (VM) on a single physical ...
A general-purpose virtualization service for HPC on cloud computing: an application to GPUs
PPAM'11: Proceedings of the 9th international conference on Parallel Processing and Applied Mathematics - Volume Part IThis paper describes the generic virtualization service GVirtuS (Generic Virtualization Service), a framework for development of split-drivers for cloud virtualization solutions. The main goal of GVirtuS is to provide tools for developing elastic ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Publisher
IGI Global
United States
Publication History
Published: 01 January 2015
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Jan 2025