MODEL EVALUATION

Why is model evaluation important?

Model evaluation determines how well climate models represent historical climate and forms an integral part of the confidence building exercise for climate change projections (Technical Report Section 6.4). Climate scientists assume (appropriately) that the better models perform over the historical period the more confidence can be assigned to their projected changes.

How do the models perform?

Broad-scale temperature, rainfall and surface wind climatologies are well captured by global climate models on seasonal and annual time scales. Some CMIP5 models were identified as being somewhat deficient across several assessments (Table 5.6.1 in Technical Report Chapter 5) and are therefore highlighted as such in the Climate Futures Tool. Similarly, a set of CMIP6 models are deficient across a range of assessments (Grose et al 2023).

The ability of global climate models to capture broad-scale climatologies is linked to their ability to reproduce the major climatic features and modes of variability affecting Australia (Technical Report Section 5.2), such as major pressure systems, seasonal and annual cycles of rainfall and temperature, the monsoon, El Niño - Southern Oscillation (ENSO) and Indian Ocean Dipole (IOD). Some of these feature and modes of variability, and their effects on the Australian climate, are slightly better represented in the current, CMIP6, generation of global models than the previous, CMIP5, generation (see Grose et al 2020). For example, most of the CMIP5 models exhibit biases in the central Pacific sea surface temperatures, which impact on ENSO and Australian rainfall. Overall, the biases in Pacific sea surface temperatures are similar between the CMIP5 and CMIP6 models. However, individual CMIP6 models show either an improvement or little change in simulating ENSO and its relationship with Australian rainfall (see Grose et al 2020).

What are the regional differences in performances?

The ability of individual global climate models to simulate Australian climate varies depending on the climatic variable, region and season under consideration. No small subset of either the CMIP5 or CMIP6 models can be identified that outperforms the remaining models consistently. However, for both CMIP5 and CMIP6, there are models that perform poorly across a number of metrics used to assess their performance. For details of model performance for the CMIP5 and CMIP6 models, see Technical Report (Section 5.2) and Grose et al (2023) respectively.

Is there low confidence for some models in some regions?

Geographically finer details around pronounced topography are difficult to simulate with a global model. As a result, several CMIP5 models show biases in regions such as Tasmania and the Cape York Peninsula (see Technical Report Section 5.2). The CMIP6 models show dry biases in rainfall in areas with pronounced topography and coastlines (see Grose el al 2020).

Does sub-setting of models according to evaluation results change rainfall projections for Australia?

Selecting a subset of better performing models did not provide more clarity on the direction of the expected rainfall changes. Using the full range of available models, rather than a performance-based selection or weighting, is the chosen method for regional applications (Technical Report Chapter 6 and Box 6.2.1). Model evaluation results, however, may influence the choice of individual models in some applications.