Model Calibration and Quantification of Predictive Uncertainty Using PEST


Dr. John Doherty (author of PEST)

October 7-11, 2013 - Neuchâtel, Switzerland

Centre for Hydrogeology,University of Neuchâtel

Model Calibration, Uncertainty Analysis and Decision-Support

The course is taught over five days. The program for most days is a combination of lectures and practical sessions (hands on exercises). The preliminary program is found below.

The content of practical sessions can be defined by individual students. A wide range of workshops is provided with the course. However students are welcome to bring their own models to use with PEST. If they do this, help in linking their models to PEST will be provided.

Day 1 is different from other days. It will be divided into two parts. In each of these parts special topics will be explored. Presenters will be John Doherty and Philip Brunner.

The morning of day 1 will be devoted to a short “refresher course” on some of the principles of linear analysis and matrix algebra which feature strongly in PEST; Dr. Philip Brunner will assist in presenting this segment of the course.  While equations will feature prominently, the accent will be on explaining their meaning rather than on rigorous analysis. Those who feel nauseous at the thought do not need to attend this segment, but we assure you that it will not be as bad as you think.

The afternoon will be devoted to an overview of the use of models in the decision-making framework. It will examine what models can and cannot offer to this framework, and analyze the strengths and weaknesses of different approaches to modelling. This will provide a context for the rest of the course. At the same time it will offer managers and those who formulate policy on the basis of models, but who do not actually use them, a chance to attend the course for a limited amount of time, and to gain maximum benefit from it. This session is open to anyone who wishes to attend, free of charge.

Day 5 is also unique. The morning session will be led by Dr Philippe Renard. It will cover stochastic approaches to model predictive uncertainty analysis. It will explain the ideas behind the many different methods used for generation of stochastic fields describing spatial variation of model properties in geologically realistic ways. It will also address the issue of how these methods can be integrated with inversion methods, so that both expert knowledge and information contained in historical measurements of system state can feature in both the analysis, and in the reduction, of model predictive uncertainty.

DAY 1 morning: Overview of linear analysis and matrix algebra

Presenters

John Doherty, Philip Brunner

Topics Covered

  • Description of matrices and vectors
  • Matrix multiplication
  • Subspaces
  • Projection operators
  • Orthogonality and orthogonal projections
  • The null space
  • Singular value decomposition
  • Eigenvalues and eigenvectors
  • Random vectors
  • Covariance matrices
  • Principal component analysis

DAY 1 afternoon: Models and Decision Support: what models can and cannot achieve

Presenters

John Doherty

Overview

The theory of traditional parameter estimation will be covered. In doing so, the strengths and weaknesses of the traditional, parsimionious approach to environmental model parameterization will be discussed. Implementation of basic parameter estimation in PEST will also be discussed. Students will then get hands-on experience in using PEST with a simple environmental model.

Topics Covered

  • An overview of modelling
  • Sources of predictive uncertainty - Bayes equation
  • The role of risk in decision-making
  • The role of uncertainty analysis in establishing risk
  • What a simple model can achieve
  • What a complex model can achieve
  • The effects of model imperfections on model predictions
  • The metrics for good modelling practice
  • What calibration does for a model
  • What calibration does not do for a model
  • When bad models can make good predictions
  • When good models make bad predictions
  • Basics of uncertainty analysis
  • Model-based hypothesis-testing
  • Using models to encapsulate what we know
  • Using models to quantify what we don’t know

DAY 2: Basics of Parameter Estimation: Theory and Practice

Presenters

John Doherty

Topics Covered

  • An overview of modelling
  • Sources of predictive uncertainty - Bayes equation
  • The role of risk in decision-making
  • The role of uncertainty analysis in establishing risk
  • What a simple model can achieve
  • What a complex model can achieve
  • The effects of model imperfections on model predictions
  • The metrics for good modelling practice
  • What calibration does for a model
  • What calibration does not do for a model
  • When bad models can make good predictions
  • When good models make bad predictions
  • Basics of uncertainty analysis
  • Model-based hypothesis-testing
  • Using models to encapsulate what we know
  • Using models to quantify what we don’t know

DAY 3 morning:  Applied groundwater parameter estimation

Presenter

John Doherty

Topics Covered

Multi-layered models

  • Steady-state and transient calibration
  • Utility support software supplied with PEST
  • Avoiding structural noise caused by model inadequacies
  • Temporal and spatial differencing in formulation of an objective function
  • Problems associated with traditional approach to calibration of groundwater models

Day 3 afternoon :  highly parameterized inversion

Presenter

John Doherty

Topics Covered

The benefits of highly parameterized inversion

  • Tikhonov regularization
  • Measurement and regularisation objective functions
  • Pilot points as a parameterization device
  • Singular value decomposition
  • Solution subspace and null subspace
  • Use of “super parameters” and “SVD-assist” for efficient inversion
  • Difference between error and uncertainty
  • Some case histories

DAY 4 morning: Applied surface water model calibration

Presenter

John Doherty

Topics Covered

  • Lumped parameter and distributed parameter models
  • Formulation of a multi-component objective function
  • Digital filtering of model outputs
  • TSPROC and other utility support software
  • Local objective function minima
  • Global optimization methods
  • Problems associated with traditional approach to calibration of SW models
  • Highly parameterized inversion in surface water and land use modelling

DAY 4 afternoon: Applied surface water model calibration

Presenter

John Doherty

Topics Covered

  • Non-uniqueness and uncertainty
  • Traditional uncertainty analysis
  • Benefits of highly parameterized uncertainty analysis
  • Linear error and uncertainty analysis
  • Nonlinear error and uncertainty analysis
  • Null space Monte Carlo
  • Pareto analysis and model-based hypothesis testing
  • Assessment of the worth of new and existing data
  • Paired simple and complex models
  • Practical examples and demonstrations

DAY 5 morning:  stochastic field generation

Presenters

Erik McCurdy

Topics Covered

  • FePEST – A User Interface for Using PEST with FEFLOW

Presenters

John Doherty and Philippe Renard

Topics Covered

  • The inherently stochastic nature of expert knowledge
  • Brief overview of geostatistics
  • Gaussian methods
  • Multiple point geostatisics
  • Other method of stochastic field generation
  • Constraining parameter fields to respect calibration constraints

DAY 5 afternoon: winding up

Presenters

John Doherty

Topics Covered

This contents of this session will be subject to participants’ wishes. It may include the following.

  • Revision of topics covered during the course
  • Further work on tutorials
  • Further demonstrations of functionality of PEST (or other software discussed during the course).
  • Discussions of topics of interest to participants

What Participants will Receive

Participants will receive a memory stick containing the following:

  • Latest version of PEST
  • Latest version of all PEST support utilities (over 200 programs)
  • Copies of files and documentation for over 12 PEST workshops
  • Literature (mainly published papers) on the use of PEST
  • Copies of all slides used in the workshop

This page has been updated the 6/1/2013