Theory of Inference (MATH 35600)
Office hours, Summer Term. These will be held:
- 1610 Thu 3 May
- 1400 Tue 8 may
- 1610 Thu 17 May
- 1610 Thu 24 May
in my office. If you are attending, please arrive at the start of
the hour with your questions.
Navigation:
Course outline,
details,
homework/assignments.
The basic premise of inference is our judgement that the things we
would like to know are related to other things that we can
measure. This premise holds over the whole of the sciences. The
distinguishing features of statistical science are
- A probabilistic approach to quantifying uncertainty, and, within
that,
- A concern to assess the principles under which we make good
inferences, and
- The development of tools to facilitate the making of such
inferences.
This course concerns (1) and (2), using the rigorous framework of
probability. In a selective approach to what is a huge area, it
weaves together the concepts of exchangeability, sufficiency, the
Exponential family, the Likelihood Principle, and Maximum Likelihood
and Bayesian inference.
Textbooks
I will base the course material on two standard reference books,
- M. Schervish, 1995, Theory of Statistics, Springer, and
- A. Davison, 2003, Statistical Modelling, Cambridge
University Press.
These are not UG textbooks! Several useful textbooks are listed on
the official
Unit page. Additionally, there is plenty of material available on-line
concerning the key concepts, and there are links given in
the details below.
There are many textbooks on statistical inference in the University Library,
and also excellent resources online. Acquiring and synthesising information
is a valuable transferable skill, and here is an opportunity to practice it.
Having said that, there will be handouts on topics that are less-well covered
in the standard textbooks. An initial handout will cover notation and
concepts.
The exam comprises three questions, of which your best two count. In my
experience it is much better to pick out your best two questions at
the start of the exam, and focus on these, than to try all three. If you
adopt the latter strategy, all of the time you put into your weakest question
is wasted.
Previous exam papers are available on Blackboard. You should be aware that
the course has changed this year, and these questions cannot be taken as a
reliable quide to the questions that will be set this year.
Answers to previous exam papers will not be made available. The
exam is designed to assess whether you have attended the lectures, read and
thought about your lecture notes and the handouts, done the homework, and
read a bit more widely in the textbooks. Diligent students who have done
the above will gain no relative benefit from studying the answers to
previous exam questions. On the other hand, less diligent students may
suffer the illusion that they will do well in the exam, when probably they
will not.
Here is a lecture-by-lecture summary of the course, looking as far ahead as
seems prudent. This plan is subject to revision.
Introduction
This material is all covered (plus extensions) in the standard textbooks, for
example
- J.A. Rice, 1999, Mathematical Statistics and Data Analysis, 2nd
edn, Duxbury Press.
- L.Wasserman, 2004, All of Statistics: A Concise Course in Statistical
Inference, Springer.
- Notation, probability. Handout
- Signficance tests, confidence intervals, Maximum Likelihood
- Bayesian inference
Sufficiency
An excellent book on the more mathematical issues, including sufficiency and
the Exponential family (below) is
- Definition and implications
- The Fisher-Neyman Factorisation Criterion
- Statistics, partitions, and minimal sufficiency. Handout
- The shape of the likelihood function is minimal sufficient
(Dynkin-Lehmann-Scheffé Theorem)
The Exponential family
The classic book on the Exponential family is
- The Exponential family of distributions
- More on the Exponential family, the Pitman-Koopmans-Darmois Theorem
Principles
For the material on principles, the following is authoritative:
- Birbaum's notion of evidential meaning. The Weak Sufficiency Principle
(WSP). Handout
- The Weak Conditionality Principle (WCP) and the Likelihood Principle
(LP). LP ⇒ { WSP & WCP }.
- { WSP & WCP } ⇒ LP, implications for inference.
- The Stopping Rule Principle (SRP), experimental ancillary statistics.
Exchangeability
- Introduction, definitions and construction. Handout
- De Finetti's Representation Theorem, generalisation (the
De Finetti-Hewitt-Savage Representation Theorem).
- Prediction with exchangeable quantities.
- Hierarchical models.
- Wrap-up.
That's it!
There will be a homework every week, set on the Friday and due back a week
later. Either hand in your homework after the Fri lecture, or leave it in the
box outside my office door (before 5pm).
Two of the homeworks will be assessed for the course credit points,
probably homeworks 2 and 5. You are strongly encouraged to do the
homeworks and to hand in your efforts, to be commented on and marked.
