Keynotes

Masashi Sugiyama
The University of Tokyo (http://www.ms.k.u-tokyo.ac.jp/sugi/index.html)

Direct Change Detection without Identification

Abstract: (slides) In this talk, we consider the problem of detecting change in two sets of samples and introduce two approaches: distributional and structural change detection. The objective of distributional change detection is to find change in probability distributions behind the two sets of samples.  This can be achieved by estimating a divergence between the probability distributions, such as the Kullback-Leibler divergence and the L2-distance.  We first explain that the two-step approach of first estimating the probability densities and then computing the divergence from the estimated densities works poorly.  We then introduce methods of directly estimating the divergence without density estimation and show their usefulness. Structural change detection aimed at finding change in pairwise dependency between elements of multi-dimensional data samples.  We first consider the Gaussian sparse covariance selection setup and explain again that separately estimating the covariance matrices from two sets of samples works poorly in structural change detection.  We then introduce our approach of directly estimating change in covariance matrices based on direct density ratio estimation. Finally, we show that our density ratio approach can be easily extended to change detection in non-Gaussian Markov networks.

Biography: Masashi Sugiyama received the PhD degree in Computer Science from Tokyo Institute of Technology in 2001 and he is currently Professor at the University of Tokyo. He received an Alexander von Humboldt Foundation Research Fellowship and researched at Fraunhofer Institute in Berlin from 2003 to 2004. In 2006, he received an European Commission Program Erasmus Mundus Scholarship and researched at the University of Edinburgh. He received the Faculty Award from IBM in 2007 for his contribution to machine learning under non-stationarity and the Young Scientists' Prize from the Commendation for Science and Technology by the Minister of Education, Culture, Sports, Science and Technology Japan for his contribution to the density-ratio paradigm of machine learning. His research interests include theories and algorithms of machine learning and data mining, and a wide range of applications such as signal processing, image processing, and robot control.

XuanLong Nguyen
Michigan University (http://dept.stat.lsa.umich.edu/~xuanlong/)

Topic modeling with more confidence -- a theory and some algorithms

Abstract: (slides) Topic models such as the latent Dirichlet allocation (LDA) have become a standard staple in the statistical modeling toolbox of data mining and machine learning. They have been applied to a vast variety of data sets, contexts, and tasks to varying degrees of success. However, to date there is almost no formal theory explicating the LDA's behavior, and despite its familiarity there is very little systematic analysis of and guidance on the properties of the data that affect the inferential performance of the model. In this talk I will report some recent progress on this front, by outlining a systematic analysis of factors which characterize the LDA's performance, and presenting a theorem elucidating the posterior contraction behavior of the topics as more data become available. I will highlight the difficulty of learning in the overfitted setting, i.e., when the number of topics in the model is larger than necessary, and then propose a LDA-based hypothesis testing framework which can be used to effectively detect new topics that emerge from text data.

Biography: XuanLong Nguyen is currently Assistant Professor in the Department of Statistics and Department of Electrical Engineering and Computer Science at the University of Michigan, Ann Arbor. He received his PhD from the University of California, Berkeley in 2007 under the supervision of Professors Michael I Jordan and Martin Wainwright, and spent two years as a postdoctoral fellow at SAMSI and Duke University's Department of Statistical Science working with Professor Alan Gelfand.

Long Nguyen is interested in Bayesian nonparametric statistics, distributed and variational inference and machine learning methods for complex and distributed systems, as well as applications in signal processing and environmental sciences. He is a recipient of the Leon O. Chua Award from UC Berkeley for his PhD research, the IEEE Signal Processing Society's Young Author best paper award, the NSF CAREER award from the Division of Mathematical Sciences, and a couple of best paper awards from the International Conference on Machine Learning (ICML) in 2004 and 2014.

Thorsten Joachims
Cornell University (http://www.cs.cornell.edu/People/tj/)

Online and Batch Learning with Interventions

Abstract: Much of the "Big Data" we collect today are the decisions that people make when using information systems. These decisions include clicks on search results, ratings of movies, purchases of products, and many more. Clearly, these decisions convey interesting information, but the actions that users take are heavily biased by interventions that the system made. For example, a search engine intervenes by presenting a specific ranking, limiting which search results the user can click on.

In this talk, I explore how interventions are both a challenge as well as an opportunity for learning. First, the ability to design and interactively schedule interventions provides great opportunities for gathering more reliable data in a goal directed way. To this effect, the talk presents online learning methods for scheduling interventions that are informed by microeconomic models of human behavior. For batch learning without interactive control, however, interventions make learning more challenging since the resulting data does not fit the assumptions of the standard machine learning models. Nevertheless, the talk shows that it is possible to derive batch learning algorithms that operate on data from logged interventions through counterfactual estimation techniques.

Biography: Thorsten Joachims is a Professor in the Department of Computer Science and the Department of Information Science at Cornell University. His research interests center on a synthesis of theory and system building in machine learning, with applications in information access, language technology, and recommendation. His past research focused on support vector machines, text classification, structured output prediction, convex optimization, learning to rank, learning with preferences, and learning from implicit feedback. In 2001, he finished his dissertation advised by Prof. Katharina Morik at the University of Dortmund. From there he also received his Diplom in Computer Science in 1997. Between 2000 and 2001 he worked as a PostDoc at the GMD Institute for Autonomous Intelligent Systems. From 1994 to 1996 he was a visiting scholar with Prof. Tom Mitchell at Carnegie Mellon University. He is an ACM Fellow, AAAI Fellow, and Humboldt Fellow.