We devise a sequential Monte Carlo method, via particle learning (PL), for on-line sampling from the posterior distribution of a sequential process over trees. The mixing problems which typically plague MCMC approaches to similar Bayesian CART models are circumvented by an automatic mixing of global (re-sample) and local(propagate) rules -- the cornerstones of PL. We consider regression trees with both constant and linear mean functions at the leaves, and thereby index a thrifty but powerful family of nonparametric regression models. We further exploit the on-line nature of inference with extensions for sequential design (a.k.a. active learning) under tree models for both (improved) surface prediction and global optimization applications. In both cases, we demonstrate how our proposed algorithms provide better results compared to higher-powered methods but use a fraction of the computational cost.
This is joint work with Matt Taddy and Nicholas Polson.