Model Calibration and Quantification of Predictive Uncertainty Using PEST


Dr. John Doherty (author of PEST)

September 5-9, 2011 - Neuchâtel, Switzerland

Stochastic Hydrogeology Group,
Centre for Hydrogeology,University of Neuchâtel

Model Calibration, Uncertainty Analysis and Decision-Support

The course is taught over five days. The program for most days is a combination of lectures and practical sessions (hands on exercises). The preliminary program is found below.

The content of practical sessions can be defined by individual students. A wide range of workshops is provided with the course. However students are welcome to bring their own models to use with PEST. If they do this, help in linking their models to PEST will be provided.

Day 1 is different from other days. This will be devoted to an overview of the use of models in the decision-making framework. It will examine what models can and cannot offer to this framework, and analyze the strengths and weaknesses of different approaches to modelling. This will provide a context for the rest of the course. At the same time it will offer managers and those who formulate policy on the basis of models, but who do not actually use them, a chance to attend the course for a limited amount of time, and to gain maximum benefit from it.

DAY 1: Models and Decision Support: What Models Can and Cannot Achieve

Presenters

John Doherty, Phillipe Renard

Overview

The day will be devoted to a series of lectures, and to discussions, on the role of modelling in the process of environmental decision-making. Particular focus will be placed on where, as an industry, we are going wrong, and how we can do better.

Topics Covered

Topics covered will include the following:

  • An overview of modelling
  • Sources of predictive uncertainty - Bayes equation
  • The role of risk in decision-making
  • The role of uncertainty analysis in establishing risk
  • What a simple model can achieve
  • What a complex model can achieve
  • The effects of model imperfections on model predictions
  • The metrics for good modelling practice
  • What calibration does for a model
  • Basics of uncertainty analysis
  • Model-based hypothesis-testing
  • Using models to encapsulate what we know
  • Using models to quantify what we don’t know

DAY 2: Basics of Parameter Estimation: Theory and Practice

Presenters

John Doherty

Overview

The theory of traditional parameter estimation will be covered. In doing so, the strengths and weaknesses of the traditional, parsimionious approach to environmental model parameterization will be discussed. Implementation of basic parameter estimation in PEST will also be discussed. Students will then get hands-on experience in using PEST with a simple environmental model.

Topics Covered

Topics covered will include the following:

  • Basic statistics - mean, standard deviation, covariance matrices
  • Theory of nonlinear parameter estimation
  • Application of nonlinear parameter estimation to model calibration
  • Parameter correlation and non-uniqueness
  • Formulation and minimization of an objective function
  • Analysis of residuals
  • The Jacobian matrix
  • Parallelization of model runs
  • Useful statistics arising from the parameter estimation process
  • The “nuts and bolts” of using PEST
  • Parameter non-uniqueness

DAY 3: Applied Parameter Estimation

Presenters

John Doherty

Overview

The use of parameter estimation in calibration of ground and surface water models will be discussed. Many practical examples will be provided. Emphasis will be placed on the “art” of parameter estimation as the theory discussed on the previous day is turned into practice.

Topics Covered

Topics covered will include the following:

Groundwater model calibration

  • Multi-layered models
  • Steady-state and transient calibration
  • Utility support software
  • Avoiding structural noise caused by model inadequacies
  • Temporal and spatial differencing in formulation of objective function
  • Problems associated with traditional approach to calibration of GW models

Surface water model calibration

  • Lumped parameter and distributed parameter models
  • Formulation of a multi-component objective function
  • Digital filtering of model outputs
  • TSPROC and other utility support software
  • Local objective function minima
  • Global optimization methods
  • Problems associated with traditional approach to calibration of SW models

DAY 4: Highly Parameterized Inversion

Presenters

John Doherty

Overview

Highly parameterized inversion as a means of overcoming problems associated with traditional approaches to model calibration and uncertainty analysis will be discussed. Practical applications of the use of this approach in groundwater and surface water modelling will be provided. The use of pilot points as a parameterization device for groundwater models will be discussed and demonstrated.

Topics Covered

Topics covered will include the following:

  • The benefits of highly parameterized inversion
  • Tikhonov regularization
  • Measurement and regularisation objective functions
  • Pilot points as a parameterization device
  • Singular value decomposition
  • Solution subspace and null subspace
  • Use of “Super parameters” and “SVD-Assist” for efficient inversion
  • Difference between error and uncertainty

DAY 5: Highly Parameterized Uncertainty Analysis

Presenters

John Doherty and Philip Brunner

Overview

A comprehensive coverage will be provided on state-of-the-art methods for analyzing the uncertainty associated with model parameters and with predictions that depend on them. Sources of model predictive uncertainty will be discussed, as will be related topics such as the contribution made to uncertainty by different aspects of a model, and the use of uncertainty analysis to inquire into means for optimal acquisition of new data to reduce uncertainty. Beo Pest and parallel PEST is used in the excersize.

Topics Covered

Topics covered will include the following:

  • Non-uniqueness and uncertainty
  • Traditional uncertainty analysis
  • Benefits of highly parameterized uncertainty analysis
  • Linear error and uncertainty analysis
  • Nonlinear error and uncertainty analysis
  • Null Space Monte Carlo
  • Pareto analysis and model-based hypothesis testing
  • Assessment of the worth of new and existing data
  • Paired simple and complex models
  • Practical examples, demonstrations and tutorials.
  • Parallel Pest and BeoPest

What Participants will Receive

Participants will receive a DVD containing the following:

  • Latest version of PEST
  • Latest version of all PEST support utilities (over 200 programs)
  • Copies of files and documentation for over 12 PEST workshops
  • Literature (mainly published papers) on the use of PEST

This page has been updated the 4/26/2011