Elicitation is the study of statistics or properties which are computable via empirical risk minimization. This has applications in understanding which loss function to use in a regression for a particular statistic or finding a surrogate loss function which is easier to optimize.
While several recent papers have approached the general question of which properties are elicitable, we suggest that this is the wrong question—all properties are elicitable by first eliciting the entire distribution or data set, and thus the important question is how elicitable. Specifically, what is the minimum number of regression parameters needed to compute the property?
Building on previous work, we introduce a new notion of elicitation complexity and lay the foundations for a calculus of elicitation. We establish several general results and techniques for proving upper and lower bounds on elicitation complexity. These results provide tight bounds for eliciting the Bayes risk of any loss, a large class of properties which includes spectral risk measures and several new properties of interest.
Joint work with Rafael Frongillo.
Jeff will evaluate the effects of different survey modes on respondents’ patterns of answers using an entropy measure of variability. While measures of centrality show little differences between face-to-face and Internet surveys, he will find strong patterns of distributional differences between these modes where Internet responses tend towards more diffuse positions due to lack of personal contact during the process and the social forces provided by that format. The results provide clear evidence that mode matters in modern survey research, and he will make recommendations for interpreting results from different modes.