Plenary Lectures

 Lerarning Sequence Kernels
 Adaptive Processing in a World of Projections
 Learning from Biology: Intelligent Response to Rare and Incongruent Events


Lerarning Sequence Kernels

Corinna Cortes Dr. Corinna Cortes
Google Research, NY, USA
Personal homepage
Homepage at Google Research


Corinna Cortes is the Head of Google Research, NY, where she is working on a broad range of theoretical and applied large-scale machine learning problems. Prior to Google, Corinna spent more than ten years at AT&T Labs - Research, formerly AT&T Bell Labs, where she held a distinguished research position. Corinna's research work is well-known in particular for her contributions to the theoretical foundations of support vector machines (SVMs) and her work on data-mining in very large data sets for which she was awarded the AT&T Science and Technology Medal in the year 2000. Corinna received her MS degree in Physics from the Niels Bohr Institute in Copenhagen and joined AT&T Bell Labs as a researcher in 1989. She received her Ph.D. in computer science from the University of Rochester in 1993. Corinna is also a competitive runner, placing third in the More Marathon in New York City in 2005, and a mother of two.

Abstract

Kernel methods are used to tackle a variety of learning tasks including classification, regression, ranking, clustering, and dimensionality reduction. The appropriate choice of a kernel is often left to the user. But, poor selections may lead to a sub-optimal performance. Instead, sample points can be used to learn a kernel function appropriate for the task by selecting one out of a family of kernels determined by the user. This talk considers the problem of learning sequence kernel functions, an important problem for applications in computational biology, natural language processing, document classification and other text processing areas. For most kernel-based learning techniques, the kernels selected must be positive definite symmetric, which, for sequence data, are found to be rational kernels. We give a general formulation of the problem of learning rational kernels and prove that a large family of rational kernels can be learned efficiently using a simple quadratic program both in the context of support vector machines and kernel ridge regression. This improves upon previous work that generally results in a more costly semi-definite or quadratically constrained quadratic program. Furthermore, in the specific case of kernel ridge regression, we give an alternative solution based on a closed-form solution for the optimal kernel matrix. We also report results of experiments with our kernel learning techniques in classification and regression tasks.


Adaptive Processing in a World of Projections

Sergios Theodoridis Professor Sergios Theodoridis
University of Athens, Greece
Personal homepage


Sergios Theodoridis is currently Professor of Signal Processing and Communications in the Department of Informatics and Telecommunications of the University of Athens. His research interests lie in the areas of Adaptive Algorithms and Communications, Machine Learning and Pattern Recognition, Signal Processing for Audio Processing and Retrieval. He is the co-editor of the book “Efficient Algorithms for Signal Processing and System Identification”, Prentice Hall 1993, the co-author of the book “Pattern Recognition”, Academic Press, 4th Ed. 2008, and the co-author of three books in Greek, two of them for the Greek Open University.

He has served as President of EURASIP and he is currently a member of the Board of Governors for the IEEE CAS Society. He is the co-author of four papers that have received best paper awards, including the IEEE Computational Society Outstanding Paper Award for the IEEE Transactions on Neural Networks. He is a member of the Greek National Council for Research and Technology and Chairman of the SP advisory committee for the Edinburgh Research Partnership (ERP). He has served as vice chairman of the Greek Pedagogical Institute and he was for four years member of the Board of Directors of COSMOTE (the Greek mobile phone operating company). He is Fellow of IET and Fellow of IEEE.

Abstract

Kernel methods have become recently the center of attention as the nonlinear counterparts of conventional linear supervised, unsupervised and semi-supervised learning algorithms. The goal of this talk is to present a general tool as an adaptation mechanism for optimization in (infinite dimensional) Reproducing Kernel Hilbert spaces (RKHS). The general setting is that of convex optimization via the powerful and elegant tool of projections. The structure of this talk evolves along the following directions:

  1. It presents in simple geometric arguments the basic principles behind convex optimization via projections in the generalized online setting. In contrast to the classical POCS theory, in our generalized methodology the number of convex sets changes in each algorithmic step, as time evolves and data samples are received.
  2. It demonstrate the methodology, using simple (mainly geometric) arguments for two case studies, of particular interest in the adaptive filtering community:
    • A generalized kernel-APA scheme derived irrespective of the differentiability or not of the respective cost function.
    • A constrained robust kernel-beamforming algorithm as an example of an adaptive constrained optimization, using robust statistics non- differentiable costs.
The focus behind both of these examples will be to demonstrate that the main effort behind our technique consists of shaping the set of constraints and the cost function in a form of convex sets. In the sequel, linear projections, in an analytic form, can readily be mobilized from a “tool box”, that has been constructed to cover most of the commonly used cost functions. Each algorithmic step consists of a sequence of projections, of linear complexity with respect to the number of unknown parameters. Our theory proves that the algorithm converges in the intersection of all (with a possible exception of a finite number of) the previous convex sets, where the required solution lies.

The work has been carried out in cooperation with Kostas Slavakis and Isao Yamada.


Learning from Biology: Intelligent Response to Rare and Incongruent Events

Misha Pavel Professor Misha Pavel
Oregon Health & Science University, USA
Personal homepage


Dr. Pavel is a Professor and a Division Head of Biomedical Engineering with a joint appointment in Biomedical Computer Science, both at Oregon Health & Science University. He is the director of the Point of Care Laboratory focused on unobtrusive monitoring and neurobehavioral assessment and modeling. He received his Ph.D. in Experimental Psychology from New York University; M.S. in Electrical Engineering from Stanford University, and B.S. in Electrical Engineering from Polytechnic Institute of Brooklyn.

During the last decade, Dr. Pavel research has been focused in the area of proactive healthcare with special emphasis on neurodegenerative diseases and aging. His current research is focused on the development of models and methodologies for the unobtrusive and continuous assessment of cognitive and health states of elders and patients with Parkinson’s diseases. This is a facet of his effort as a director of the Point-of-Care Laboratory to use technology to enable the transition from today’s reactive to future proactive and distributed healthcare. This includes techniques for early detection and diagnosis of neurological diseases as well as mitigation of cognitive decline. In a related work, Dr. Pavel is developing principled approaches to amplify human cognitive abilities that would enable operators to improve the quality and speed of their decision-making under uncertainty.

On the academic side, Dr Pavel’s current research is at the intersection of computational modeling of complex behaviors of biological systems, engineering and cognitive science, with a focus on information fusion, pattern recognition, augmented cognition and the development of multimodal and perceptual human-computer interfaces. Dr. Pavel developed a number of quantitative and computational models of perceptual and cognitive processes, eye movement control, and a theoretical framework for knowledge representation. The resulting models have been applied in a variety of areas ranging from computer assisted instruction systems, enhanced vision systems for aviation to augmented cognition systems.

Abstract

The generation of an appropriate response, e.g., classification, to “novel” or “rare” stimuli t is a fundamental property of any intelligent system. The generation of responses to such stimuli by natural and artificial systems has, therefore, been the focus of extensive research in cognitive science (psychology), neuroscience, computer science and engineering. Despite the advances in machine learning and statistical pattern recognition, the human perceptual and cognitive pattern recognition system is still significantly more effective in dealing with novelty, uncertainty and changing environmental conditions. In this presentation, I will illustrate examples of situations in which humans perform well, but machine recognition performance is compromised. This difference between natural and artificial systems suggests biologically-motivated, neuromorphic computational approaches that may approximate the performance of the human cognitive system. I will discuss a recently developed theoretical framework that defines novelty in terms of incongruence of features defined within naturally arising hierarchical representations in terms of divergence in probability distributions. Finally, I will illustrate the effectiveness of this approach using two simple examples.