IUK Radiosonde Analysis Project -- new V2 now available!

Background

Analyses of radiosonde, surface, and satellite temperature trends have produced discordant results, which have caused some to question the reliability of our current estimates of global warming. Recently, some convergence seems to be occurring between the satellite and ground-based data for the period of overlap. More importantly, however, the period since 1980 or so has shown different trends in lapse rate (related to atmospheric convective instability) than the period prior to that. Theories of climate predict that tropical lapse rates should become slightly shallower with warmer surface temperatures due to pseudoadiabatic convective mixing, while extratropical lapse rates should be related to the equator-to-pole temperature difference due to slantwise eddy mixing. Anomalies from theory should decay rapidly (days in the Tropics, weeks in the extratropics). The observed changes, particularly a shift in the Tropics near 1976, seem to be at odds with these theories.

Map

This figure shows changes in lower-troposheric lapse rate in several latitude bands from
87 radiosonde stations; red curves are prior to manual removal of estimated heterogeneitie
by Lanzante et al. 2003b (their Fig. 7b).

A serious impediment to accurate detection of long-term climate signals in the radiosonde network is the known presence of artificial discontinuities due to unknown changes in instrumentation and data processing. A second problem is the sporadic operation of many stations. Estimates of long-term changes are sensitive to both problems. Currently, trend estimates are quoted based on the best data available and caveats are given regarding these problems. Small subsets (<100) of the available (~1000) stations are usually used to reduce these problems, leading to suboptimal spatial coverage, but still without eliminating the problems.

The Iteratively Homogenized Dataset version 2.0

At long last we introduce the revised version of the IUK dataset, now containing data through Feb. 2013. Links to the new data appear after the original version in the next section.

The updated dataset is prepared using the same methodology as the original version, and on the same stations, but with three modifications:
  • We now use straight wind vector data rather than wind shear. This permits us to produce a homogenised wind dataset which was not available previously. Unexpectedly, the use of winds also seems to have had a beneficial effect on the temperature estimates, as the trends more closely resemble expectations. This is described in a manuscript still in preparation.
  • Two small bugs were fixed but were not observed to have a significant impact on results.
  • Data are now available at all mandatory levels from 850 hPa to 30 hPa. However, we no longer provide "B" station data whose homogeneity cannot be obtained as confidently as for "A" stations.
  • Structural basis used to represent natural variability within the IUK iterative fitting algorithm now includes a cubic polynomial, which aids in capturing decadal variations more accurately.
It is anticipated that we will issue updated versions of the dataset annually (although, we will see....)

The Iteratively Homogenized Dataset (original description)

We have produced a new dataset following the iterative universal Kriging procedure described in Sherwood (1999) and Sherwood (2007). The key to this approach is to estimate the climate signals, missing data, and instrument change effects synergistically, i.e., iteratively, exploiting the spatial and temporal coherence of natural variability. The resulting dataset includes temperature and wind shear at mandatory reporting levels from 527 radiosonde stations, from 1959-2005. The stations are divided into two groups: 460 A stations having substantial data at two times of day, and 67 B stations not having these (B stations are included only in the Tropics and southern hemisphere). Trends in A stations are more reliable.

The procedure appears to have been successful in eliminating systematic temperature biases in most regions, although the deep tropics appear to retain cooling biases over time that we still cannot identify; these may be due to changes that are too numerous to detect, or not step-like. A penalty paid for the elimination of systematic biases is that random errors are not reduced as effectively as other methods, so that individual stations now have trends that are about as variable as in the raw data; however, the accuracy of zonal means appears to be significantly improved. The dataset and some trend results are described more fully in Sherwood et al. (2008) .

Wind homogenization had only a small effect, so we are not proposing that our homogenized wind data are significantly better than the raw data. We interpret this as an indication that significant shifts in wind shear are isolated and not a serious problem across the network, although further studies should examine this issue more carefully. Also, this statement does not automatically apply to wind speed or direction per se, only to vertical vector wind shear, the variable that was actually homogenized.

The data are available in two NetCDF files. The 460 A stations are first, followed by the 67 B stations. In adjustment files, these two groups are contained in separate variables.

Please see the README file for further details. To obtain the files below, right-click on the name (sorry, our server doesn't allow ftp so it's done w/http). I also have a page providing software for iterative universal Kriging.

  • all_latlon.dat (11 kB), (in ASCII format) coordinates of all stations. This information is also included in each of the other files below so you shouldn't need it if you're getting the data.
  • T_monthly.nc (36 MB), X_monthly.nc (71 MB): the monthly and diurnal mean homogenized temperature (T) and windshear (X) data. Each value is the mean of all observed and imputed values for that month, with estimated artifacts subtracted out. Also included are one-sigma sampling and structural uncertainties for each data point.
  • T_CP.nc (6 MB): the change point times and level shifts found by each of two schemes (2PH for two-phase regression, L96 for nonparametric) at each of two station groups (A and B). Two types of change points are included: those affecting all times of day equally ("non-solar"), and "solar" CPs that affect only "daytime" observations (those at whichever of the two nominal observing times falls between 600 and 1800 hours local non-daylight time). Non-solar CPs are found for both station groups, while solar CPs are found only for Group A. Each non-solar CP is assumed to affect all levels, but with different shift amplitudes for each level and season; at any level/season where this is not possible due to inadequate data, the shift is set to zero. Solar CPs are defined separately for each level, but their level shifts are estimated once for all seasons. Note that a negative value for the level shift indicates a downward artifact in temperature. Artifacts should be corrected by adding the shift value to all data prior to the CP date.
  • T_monthly.nc (43 MB), UV_monthly_V2.nc (86 MB), T_CP_V2.nc (6.7 MB): the monthly and diurnal mean homogenized temperature (T) and wind (U,V) data, and T changepoint data, from IUK version 2.0. Each value is the mean of all observed and imputed values for that month, with estimated artifacts subtracted out. Also included are one-sigma sampling and structural uncertainties for each data point. Details on change-point file identical to version 1 described above.

Some literature relevant to this project

  • Sherwood, S. C. and N. Nishant, Manuscript in preparation to describe IUKv2.
  • Sherwood, S. C., H. A Titchner, P. W. Thorne and M. McCarthy, How do we tell which estimates of past climate change are correct? International Journal of Climatology, online, 2009, DOI: 10.1002/joc.1825. reprint
  • Allen, R. J. and S. C. Sherwood, Warming maximum in the tropical upper troposphere deduced from thermal wind observations. Nature Geosci., Vol. 65, 2008, 399-403. published version
  • S. C. Sherwood, C. L. Meyer, R. J. Allen, and H. A. Titchner, Robust tropospheric warming revealed by iteratively homogenized radiosonde data. Journal of Climate, Vol. 21, 2008, 5336-5352. view abstract / preprint
  • Allen, R. J. and S. C. Sherwood, Utility of radiosonde wind data in representing climatological variations of tropospheric temperature and baroclinicity in the western tropical Pacific, Journal of Climate, Vol. 20, 2007, 5229-5243. view abstract / published version
  • Sherwood, S. C., Simultaneous detection of climate change and observing biases in a network with incomplete sampling. Journal of Climate, Vol. 20, 2007, 4047-4062. view abstract / preprint / published version
  • Sherwood, S. C., J. R. Lanzante and C. L. Meyer, Radiosonde daytime biases and late 20th century warming, Science, Vol. 309, 2005, pp. 1556-1559. reprint
  • Angell, J. K., Effect of Exclusion of Anomalous Tropical Stations on Temperature Trends from a 63-Station Radiosonde Network, and Comparison with Other Analyses. J. Climate, Vol. 16, No. 13, 2003, pp. 2288-2295. reprint
  • Hegerl, G. C. and J. M. Wallace, Influence of Patterns of Climate Variability on the Difference between Satellite and Surface Temperature Trends. J. Climate, Vol. 15, 2002, 2412-2428. reprint
  • Gaffen, D. et al. Multidecadal changes in the vertical temperature structure of the tropical troposphere. Science, Vol. 287, 18 Feb. 2000, pp 1242-45. download pdf
  • Free, M. et al. Creating climate reference datasets: CARDS workshop on adjusting radiosonde temperature data for climate monitoring. Bull. Amer. Meteor. Soc., Vol. 83, No. 6, 2002, pp. 891-899.download pdf
  • Lanzante, J. R., S. A. Klein and D. J. Seidel. Temporal homogenization of monthly radiosonde temperature data. Part I: Methodology. J. Climate, Vol. 16, No. 2, 2003, pp. 224-240.download pdf
  • Lanzante, J. R., S. A. Klein and D. J. Seidel. Temporal homogenization of monthly radiosonde temperature data. Part II: Trends, Sensitivities, and MSU Comparison. J. Climate, Vol. 16, No. 2, 2003, pp. 241-262. download pdf
  • Sherwood, S. C., Climate-signal mapping and an application to atmospheric tides. Geophysical Research Letters, Vol. 27, No. 21, 2000, pp. 3525-3528. view abstract / download pdf
  • Climate signals from station arrays with missing data, and an application to winds. Journal of Geophysical Research, Vol. 105, No. D24, 2001, pp. 29,489-29,500. view abstract / download pdf
Last updated 3/25/2008.UPDATE