U.S. Department of Energy Office of Biological and Environmental Research

BER Research Highlights

New Techniques for Filling Gaps in Instrument Records
Published: February 15, 2015
Posted: May 29, 2015

Best-estimate annual total cloud fraction from radar (black) and radar + lidar (red) observations, filled with information from the SOM technique. 95% confidence intervals are given by the grey and pink shading. [Graph courtesy Kennedy, Dong, and Xi. 2015. DOI: 10.1007/s00704-015-1384-3]

The Atmospheric Radiation Measurement (ARM) Climate Research Facility’s Southern Great Plains (SGP) site in Lamont, Oklahoma, is home to one of the longest records of actively sensed cloud information anywhere in the world. Despite the best efforts of facility staff, however, instruments occasionally fail or are taken down for maintenance, resulting in holes within the observational record.  These gaps lead to uncertainty in monthly statistics of observed variables such as cloud fraction that are often used to evaluate model simulations or diagnose trends in the observations.

Researchers funded by the Atmospheric System Research (ASR) program used a statistical technique called self-organizing maps (SOM) to reduce uncertainties in the instrument record. The analysis took advantage of the fact that cloud occurrence is partly controlled by the large-scale environment and that the long time series of ARM measurements allows robust classification into meteorological regimes. Testing a number of SOM configurations, the analysis showed that uncertainty in the monthly total cloud fraction record can be reduced significantly and that the largest gain is provided by SOMs that have a large number of classes and separate data by month.  Using the new technique, uncertainty in monthly total cloud fraction was reduced in half from previous values.

This proof-of-concept work opens the door to a number of other opportunities. The methodology is adaptable to other ARM sites. Further, the results suggest that a combination of ARM observations and reanalyses can provide a better historical record of cloud occurrence prior to the existence of actively sensed observations. Finally, this work can move beyond cloud fraction and the techniques can be applied to other variables such as records of specific cloud types.

Reference: Kennedy, A. D., X. Dong, and B. Xi. 2015. "Cloud Fraction at the ARM SGP Site: Reducing Uncertainty with Self-Organizing Maps," Theoretical and Applied Climatology, DOI: 10.1007/s00704-015-1384-3. (Reference link)

Contact: Sally McFarlane, SC-23.1, (301) 903-0943
Topic Areas:

  • Research Area: Atmospheric System Research
  • Facility: DOE ARM User Facility

Division: SC-23.1 Climate and Environmental Sciences Division, BER


BER supports basic research and scientific user facilities to advance DOE missions in energy and environment. More about BER

Recent Highlights

Aug 24, 2019
New Approach for Studying How Microbes Influence Their Environment
A diverse group of scientists suggests a common framework and targeting of known microbial processes [more...]

Aug 08, 2019
Nutrient-Hungry Peatland Microbes Reduce Carbon Loss Under Warmer Conditions
Enzyme production in peatlands reduces carbon lost to respiration under future high temperatures. [more...]

Aug 05, 2019
Amazon Forest Response to CO2 Fertilization Dependent on Plant Phosphorus Acquisition
AmazonFACE Model Intercomparison. The Science Plant growth is dependent on the availabi [more...]

Jul 29, 2019
A Slippery Slope: Soil Carbon Destabilization
Carbon gain or loss depends on the balance between competing biological, chemical, and physical reac [more...]

Jul 15, 2019
Field Evaluation of Gas Analyzers for Measuring Ecosystem Fluxes
How gas analyzer type and correction method impact measured fluxes. The Science A side- [more...]

List all highlights (possible long download time)