U.S. Department of Energy Office of Biological and Environmental Research

BER Research Highlights

Framework for Managing Ultra-Large Climate Datasets
Published: March 27, 2013
Posted: June 20, 2013

Fueled by exponential increases in computational and storage capabilities of high-performance computing platforms, climate model simulations are evolving toward higher numerical fidelity, complexity, volume, and dimensionality. Data holdings are projected to reach hundreds of exabytes worldwide by 2020. Such explosive growth presents both challenges and opportunities for scientific breakthroughs. A U.S. Department of Energy (DOE) funded project, Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT), is addressing these challenges, with a team from four DOE laboratories (Lawrence Berkeley, Lawrence Livermore, Los Alamos, and Oak Ridge); two universities (Polytechnic Institute of New York University and University of Utah); National Aeronautics and Space Administration at Goddard Space Flight Center; and two private companies (Kitware and Tech-X). UV-CDAT software tools address: 1) problems with “big data” analytics; 2) the need for reproducibility; 3) requirements to push ensemble analysis, uncertainty quantification, and metrics computation to new boundaries; 4) heterogeneous data sources (simulations, observations, and re-analysis); and 5) provision of an overall architecture for incorporating existing and future software components. The team designed a Python-based framework that integrates several disparate technologies under one infrastructure. United by standard common protocols and application programming interfaces, UV-CDAT integrates more than 40 different software components. The primary goal of this nationally coordinated effort is to build an ultrascale data analysis and visualization system empowering scientists to engage in new and exciting data exchanges, thus enabling breakthrough climate science. The framework is established to evolve and incorporate the new software tools that the science and scientific community require.

Reference: Williams, D. N., T. Bremer, C. Doutriaux, J. Patchett, G. Shipman, B. Haugen, R. Miller, B. Smith, C. Steed, E. W. Bethel, H. Childs, H. Krishnan, M. Wehner, C. T. Silva, E. Santos, D. Koop, T. Ellqvist, H. T. Vo, J. Poco, B. Geveci, A. Chaudhary, A. Bauer, A. Pletzer, D. Kindig, G. L. Potter, and T. P. Maxwell. 2013. “The Ultrascale Visualization Climate Data Analysis Tools: Data Analysis and Visualization for Geoscience Data”, IEEE Computer Special Issue: Cutting-Edge Research in Visualization, in press. DOI: 10.1109/MC.2013.119. (Reference link)

Website: https://uv-cdat.llnl.gov/

Contact: Renu Joseph, SC-23.1, (301) 903-9237, Dorothy Koch, SC-23.1, (301) 903-0105
Topic Areas:

  • Research Area: Earth and Environmental Systems Modeling
  • Cross-Cutting: Scientific Computing and SciDAC

Division: SC-23.1 Climate and Environmental Sciences Division, BER


BER supports basic research and scientific user facilities to advance DOE missions in energy and environment. More about BER

Recent Highlights

May 10, 2019
Quantifying Decision Uncertainty in Water Management via a Coupled Agent-Based Model
Considering risk perception can improve the representation of human decision-making processes in age [more...]

May 09, 2019
Projecting Global Urban Area Growth Through 2100 Based on Historical Time Series Data and Future Scenarios
Study provides country-specific urban area growth models and the first dataset on country-level urba [more...]

May 05, 2019
Calibrating Building Energy Demand Models to Refine Long-Term Energy Planning
A new, flexible calibration approach improved model accuracy in capturing year-to-year changes in bu [more...]

May 03, 2019
Calibration and Uncertainty Analysis of Demeter for Better Downscaling of Global Land Use and Land Cover Projections
Researchers improved the Demeter model’s performance by calibrating key parameters and establi [more...]

Apr 22, 2019
Representation of U.S. Warm Temperature Extremes in Global Climate Model Ensembles
Representation of warm temperature events varies considerably among global climate models, which has [more...]

List all highlights (possible long download time)