U.S. Department of Energy Office of Biological and Environmental Research

BER Research Highlights

Framework for Managing Ultra-Large Climate Datasets
Published: March 27, 2013
Posted: June 20, 2013

Fueled by exponential increases in computational and storage capabilities of high-performance computing platforms, climate model simulations are evolving toward higher numerical fidelity, complexity, volume, and dimensionality. Data holdings are projected to reach hundreds of exabytes worldwide by 2020. Such explosive growth presents both challenges and opportunities for scientific breakthroughs. A U.S. Department of Energy (DOE) funded project, Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT), is addressing these challenges, with a team from four DOE laboratories (Lawrence Berkeley, Lawrence Livermore, Los Alamos, and Oak Ridge); two universities (Polytechnic Institute of New York University and University of Utah); National Aeronautics and Space Administration at Goddard Space Flight Center; and two private companies (Kitware and Tech-X). UV-CDAT software tools address: 1) problems with “big data” analytics; 2) the need for reproducibility; 3) requirements to push ensemble analysis, uncertainty quantification, and metrics computation to new boundaries; 4) heterogeneous data sources (simulations, observations, and re-analysis); and 5) provision of an overall architecture for incorporating existing and future software components. The team designed a Python-based framework that integrates several disparate technologies under one infrastructure. United by standard common protocols and application programming interfaces, UV-CDAT integrates more than 40 different software components. The primary goal of this nationally coordinated effort is to build an ultrascale data analysis and visualization system empowering scientists to engage in new and exciting data exchanges, thus enabling breakthrough climate science. The framework is established to evolve and incorporate the new software tools that the science and scientific community require.

Reference: Williams, D. N., T. Bremer, C. Doutriaux, J. Patchett, G. Shipman, B. Haugen, R. Miller, B. Smith, C. Steed, E. W. Bethel, H. Childs, H. Krishnan, M. Wehner, C. T. Silva, E. Santos, D. Koop, T. Ellqvist, H. T. Vo, J. Poco, B. Geveci, A. Chaudhary, A. Bauer, A. Pletzer, D. Kindig, G. L. Potter, and T. P. Maxwell. 2013. “The Ultrascale Visualization Climate Data Analysis Tools: Data Analysis and Visualization for Geoscience Data”, IEEE Computer Special Issue: Cutting-Edge Research in Visualization, in press. DOI: 10.1109/MC.2013.119. (Reference link)

Website: https://uv-cdat.llnl.gov/

Contact: Renu Joseph, SC-23.1, (301) 903-9237, Dorothy Koch, SC-23.1, (301) 903-0105
Topic Areas:

  • Research Area: Earth and Environmental Systems Modeling
  • Cross-Cutting: Scientific Computing and SciDAC

Division: SC-23.1 Climate and Environmental Sciences Division, BER


BER supports basic research and scientific user facilities to advance DOE missions in energy and environment. More about BER

Recent Highlights

Aug 24, 2019
New Approach for Studying How Microbes Influence Their Environment
A diverse group of scientists suggests a common framework and targeting of known microbial processes [more...]

Aug 08, 2019
Nutrient-Hungry Peatland Microbes Reduce Carbon Loss Under Warmer Conditions
Enzyme production in peatlands reduces carbon lost to respiration under future high temperatures. [more...]

Aug 05, 2019
Amazon Forest Response to CO2 Fertilization Dependent on Plant Phosphorus Acquisition
AmazonFACE Model Intercomparison. The Science Plant growth is dependent on the availabi [more...]

Jul 29, 2019
A Slippery Slope: Soil Carbon Destabilization
Carbon gain or loss depends on the balance between competing biological, chemical, and physical reac [more...]

Jul 15, 2019
Field Evaluation of Gas Analyzers for Measuring Ecosystem Fluxes
How gas analyzer type and correction method impact measured fluxes. The Science A side- [more...]

List all highlights (possible long download time)