Christian Robert, CEREMADE, Université Paris Dauphine & CREST, INSEE, Paris

Title: Adaptive Multiple Importance Sampling (AMIS)

Abstract :

The AMIS algorithm consists of 3 steps.  In the initialization step a set of iid uniform points are sampled in the d-dimensional unit box. The logistic transformation with scale parameter S, is used to bring the points back to $R^d$. The scale parameter is chosen using the ESS of the importance weights. Importance sampling estimates of the target mean and variance are constructed. These estimates are used to generate, from a d-dimensional Student-T(3df), the initial set of particles of the second adaptive step.  At this stage a temporal dimension is introduced and global adaptation is performed by an importance sampling version of Haario et al. (2001) adaptive Metropolis-Hastings algorithm. To achieve variance reduction an adaptive version of Owen and Zhou (2000), deterministic mixture importance sampling is defined.  As a byproduct of the mixture and actualization process we performe on the weights, all particles are on the same ``weighting scale'' and can be easily and efficiently combined to get final AMIS estimator. In the final step a Rao-Blackwellised  clustering algorithm is performed on all generated particles. The number of clusters, K, is chosen via BIC.  IS estimators of mean and covariance matrices on each cluster are derived.  A K-mixture of Student-T (3df) distributions is used to  generate  a final set of particles.  The mixture proportions are taken to be the number of particles belonging to each cluster. The AMIS estimator is obtained by recycling the particles generated in all 3 steps, with the corresponding importance weights.  The strength of AMIS resides in its completely adaptive and multi-purpose nature: no tuning parameter is needed and the same algorithm is proved to perform well on very diverse high-dimensional target distributions (from banana shaped to mixture with very well separated modes).

(This is joint work in progress with Jean-Michel Marin and Antonietta Mira)