ENGS 299: Advanced Topics in Computational Science


Advanced Statistical Analysis of Experimental Data:

Statistical Analysis of Images


Fall 2000

Thayer School of Engineering


Eugene Demidenko, PhD

Section of Biostatistics and Epidemiology at Dartmouth Medical School

and Department of Mathematics at Dartmouth College (NH)


Where and when: Room 202 Cummings, Monday&Thursday 3-4:45.

Homework to be handout on Thursday and must be returned next Monday.



My office: Lebanon, DHMC, Norris Cotton Cancer Center, Room 876. TEL (603) 653-3682.

My e-mail: Eugene.demidenko@dartmouth.edu


Course description. Modern data of engineering problems, such as signal processing or images, require advanced methods of statistical analysis. The goal of this course is to introduce several adequate and recently emerged statistical techniques not covered by traditional statistical courses, such as statistical analysis of images.  The role of optimization methods for inverse and ill-posed problems is emphasized. We will start off with a review of general statistical principals for point and interval estimation, and hypothesis testing. Then we will move to nonlinear regression models and optimization methods with an emphasis on the inverse problem solution derived from differential equations. Linear and nonlinear regression models will illustrate Bayesian approach. Links between Bayesian approach and optimization of ill-posed optimization problem and Tikhonov regularization is discussed. Kalman filter and Markov random field theory will be the focus of our attention as powerful tools in signal processing and image reconstruction.


Prerequisites: ENGS 91, ENGS 103, ENGS 105


Syllabus. The course consists of two modules and each module will be covered by a take-home exam. The homework will be assigned weekly.  In the team project (2 people per project) topics will be chosen close to scientific interests of participants. Team project will involve application of existing software packages such as Matlab/S-Plus or programming in C/Fortran. We anticipate that by doing a team project students will learn how to apply the gained knowledge within a stand-alone scientific activity.


We will have two classes per week; the instructor may provide private consultations as well.


The final course grade is broken down as follows:


Take home exam (module 1)………..25%

Take home exam (module 2)………..25%

Team project………………………...20%



The following topics will be covered (1-2 weeks per item):


1.      Principles of statistical approach: point and interval estimation, and hypothesis testing. Method of maximum likelihood and its properties. Least squares and linear regression model.

2.      Linear regression model, ill-posed regression problem (matrix approach). Design of experiments for linear model. Solving inverse problem by the least squares.

3.      Principles of unconstrained optimization. Nonlinear regression model and minimization of sum of squares by Gauss-Newton and Levenberg-Marquardt model. Stopping criteria. Estimation of implicit relationships specified by differential equations.

4.      Bayesian analysis and penalized least squares. Tikhonov regularization. Mixed effects and hierarchical modeling and its application to solve ill-posed problems.

Take-home exam on topics 1-4.


5.      Statistical issues of signal processing and Kalman filter. Optimal estimation and statistical inference. Sensitivity analysis to measurement errors. Elements of statistical analysis for MRI and fMRI.

6.      Spatial statistics and image reconstruction. Statistical analysis of images. Markov random fields. Elements of computed and positron emission tomography (CT and PET). Principles of optimal scanning and statistical inference.

7.      Modern nonparametric techniques in regression analysis and density estimation, local and robust regression. Principal components and cluster analyses, and classification. Modern multivariate statistical inference and multidimensional scaling.

Take-home exam on topics 5-7.



There is no one single text. Below is some literature we will use in our course.


Rice JA (1995). “Mathematical Statistics and Data Analysis”. Duxbury Press. Belmont. This is one of solid standard texts on principles of statistical inference with many examples from physics and engineering.


Draper NR and Smith H (1998). “Applied Regression Analysis”, John Wiley, New York. Standard text on regression analysis with some good examples from physics and chemistry. A good introduction to nonlinear estimation.


Seber GAF and Wild CJ (1989). “Nonlinear Regression”, John Wiley, New York.  Complete references book to nonlinear regression problems with some engineering applications.


Bates DM and Watts DG (1989). “Nonlinear regression analysis and its applications”. John Wiley. New York. Another good book on nonlinear regression with emphasis on statistical inference and confidence intervals computation.


Kamen EW and Su JK (1999). “Introduction to Optimal Estimation”, Springer, London. In this book Kalman filter and statistical issues of signal processing are discussed in detail.


Ripley BD (1981). “Spatial Statistics” John Wiley, New York. This is a standard reference book for statistical analysis on the plane or spatial statistics. We will use it when studying image processing.


Cressie NAC (1991). “Statistics for Spatial Data”. A more advanced book on spatial statistics, there is a special chapter on image analysis.


Fukunaga, Keinosuke (1990). “Introduction to Statistical Pattern Recognition”, 2nd Edn, Academic Press.



Visit my WEB pages:

http://www.math.dartmouth.edu/~m33s99 – Mathematics for Sciences and Engineering, M33

http://www.math.dartmouth.edu/~m50w99 – Probability and Statistical Inference, M50

http://www.dartmouth.edu/~eugened – Webpage of my book “Mixed Models: Theory and Applications”