BER launches Environmental System Science Program. Visit our new website under construction!

U.S. Department of Energy Office of Biological and Environmental Research

BER Research Highlights

Framework for Managing Ultra-Large Climate Datasets
Published: March 27, 2013
Posted: June 20, 2013

Fueled by exponential increases in computational and storage capabilities of high-performance computing platforms, climate model simulations are evolving toward higher numerical fidelity, complexity, volume, and dimensionality. Data holdings are projected to reach hundreds of exabytes worldwide by 2020. Such explosive growth presents both challenges and opportunities for scientific breakthroughs. A U.S. Department of Energy (DOE) funded project, Ultrascale Visualization Climate Data Analysis Tools (UV-CDAT), is addressing these challenges, with a team from four DOE laboratories (Lawrence Berkeley, Lawrence Livermore, Los Alamos, and Oak Ridge); two universities (Polytechnic Institute of New York University and University of Utah); National Aeronautics and Space Administration at Goddard Space Flight Center; and two private companies (Kitware and Tech-X). UV-CDAT software tools address: 1) problems with “big data” analytics; 2) the need for reproducibility; 3) requirements to push ensemble analysis, uncertainty quantification, and metrics computation to new boundaries; 4) heterogeneous data sources (simulations, observations, and re-analysis); and 5) provision of an overall architecture for incorporating existing and future software components. The team designed a Python-based framework that integrates several disparate technologies under one infrastructure. United by standard common protocols and application programming interfaces, UV-CDAT integrates more than 40 different software components. The primary goal of this nationally coordinated effort is to build an ultrascale data analysis and visualization system empowering scientists to engage in new and exciting data exchanges, thus enabling breakthrough climate science. The framework is established to evolve and incorporate the new software tools that the science and scientific community require.

Reference: Williams, D. N., T. Bremer, C. Doutriaux, J. Patchett, G. Shipman, B. Haugen, R. Miller, B. Smith, C. Steed, E. W. Bethel, H. Childs, H. Krishnan, M. Wehner, C. T. Silva, E. Santos, D. Koop, T. Ellqvist, H. T. Vo, J. Poco, B. Geveci, A. Chaudhary, A. Bauer, A. Pletzer, D. Kindig, G. L. Potter, and T. P. Maxwell. 2013. “The Ultrascale Visualization Climate Data Analysis Tools: Data Analysis and Visualization for Geoscience Data”, IEEE Computer Special Issue: Cutting-Edge Research in Visualization, in press. DOI: 10.1109/MC.2013.119. (Reference link)


Contact: Renu Joseph, SC-23.1, (301) 903-9237, Dorothy Koch, SC-23.1, (301) 903-0105
Topic Areas:

  • Research Area: Earth and Environmental Systems Modeling
  • Cross-Cutting: Scientific Computing and SciDAC

Division: SC-33.1 Earth and Environmental Sciences Division, BER


BER supports basic research and scientific user facilities to advance DOE missions in energy and environment. More about BER

Recent Highlights

Mar 23, 2021
Molecular Connections from Plants to Fungi to Ants
Lipids transfer energy and serve as an inter-kingdom communication tool in leaf-cutter ants&rsqu [more...]

Mar 19, 2021
Microbes Use Ancient Metabolism to Cycle Phosphorus
Microbial cycling of phosphorus through reduction-oxidation reactions is older and more widespre [more...]

Feb 22, 2021
Warming Soil Means Stronger Microbe Networks
Soil warming leads to more complex, larger, and more connected networks of microbes in those soi [more...]

Jan 27, 2021
Labeling the Thale Cress Metabolites
New data pipeline identifies metabolites following heavy isotope labeling.

Analysis [more...]

Aug 31, 2020
Novel Bacterial Clade Reveals Origin of Form I Rubisco

  • All plant biomass is sourced from the carbon-fixing enzyme Rub [more...]

List all highlights (possible long download time)