Benchmarks of Realistic Scientific Application Performance of Large-Scale Computing Sys... - 0 views
-
MiamiOH OARS on 25 Nov 14NSF is interested in supporting activities by the NSF Cyberinfrastructure community in the analysis of existing benchmarks, and in the development of new benchmarks, that measure real-world performance and effectiveness of large-scale computing systems for science and engineering discovery. Research, development, and use of performance benchmarks in high-performance computing (HPC) has been active for over 20 years, as evidenced by the development of LINPACK and the emergence of the TOP500 list in the early 1990s, followed by the development of the HPC Challenge Benchmark and the current HPCG effort (http://tiny.cc/hpcg). There have been efforts to provide benchmarks that include real applications, such as the SPEC High Performance Computing Benchmarks (http://spec.org/benchmarks.html#hpg), the Blue Waters SPP suite (http://www.ncsa.illinois.edu/assets/pdf/news/BW1year_apps.pdf), and the NERSC SSP (https://www.nersc.gov/users/computational-systems/nersc-8-system-cori/nersc-8-procurement/trinity-nersc-8-rfp/nersc-8-trinity-benchmarks/ssp/). Recent efforts have sought to broaden the set of relevant benchmarks to more effectively cover performance under different application environments such as data-intensive analysis (e.g., Graph500). Energy efficiency has also emerged in recent years as a relevant and increasingly important area of measurement and profiling for HPC systems (e.g., Green500). In addition to HPC, the Big Data community has gained interest in benchmarking; reference approaches to measuring and characterizing system performance for large-scale data analysis hardware and software systems remains an area of research, development, and community discussion (e.g., on the Big Data Top 100). Industry and academe have convened an ongoing series of workshops and meetings on the topic of Big Data benchmarking (http://clds.ucsd.edu/bdbc/workshops). Given the emergence of inference-based computing, the growing role of data analysis, changes in scientific workflow du