Thank you to the following supporters of ICASSP 2007

TUT-2: Machine Learning and Signal Processing Applications of Fixed Point Theory

Sunday Afternoon, April 15
14:00 - 17:00
Room 324

Presented by

Danilo Mandic, Imperial College London and Isao Yamada, Tokyo Institute of Technology

Abstract

This tutorial is aimed at providing a unifying and rigorous basis for linking of different parts of signal processing and related disciplines under one umbrella. Case studies within this tutorial will address problems from several overlapping areas: nonlinear and linear adaptive filtering, image processing, estimation theory, communications, automatic control, and machine learning.

Topics which will be covered in this tutorial include:

Danilo Mandic:

Background theory and motivation
- History of fixed point theory
- Modern fixed point theory
- Random contraction mapping theorems
- Scope and perspective
Unifying framework for the analysis of linear adaptive filtering problems
- Contraction Mapping Theorem (CMT) and Fixed Point Iteration (FPI) as a tool for the unifying analysis and interpretation of the adaptive filtering problem
- Application in standard acoustic echo cancellation
- Applications in relaxation of autonomous systems
Analysis of nonlinear problems within the framework of fixed point theory
- Feedback, nonlinearity, recurrence and adaptivity
- Operation and stability of nonlinear adaptive filters
- Nonlinear dynamics, modularity, nesting, bifurcations, and chaos
Contraction mapping interpretation of learning algorithms in machine learning and adaptive filtering
- Data reusing and the trade-off between recursive and iterative learning
- Blind source separation and extraction
- Statistical signal processing: an interpretation of the EM algorithm
- Turbo decoding
Two dimensional problems: Fixed point theory for image processing and pattern recognition
- Colour image retrieval and object recognition
- Projection on convex sets (POCS) and image restoration
Tracking and stability issue in autonomous systems
- Trajectory tracking as an example of fixed point theory
- Learning dynamics, association and generalisation

Isao Yamada

Fixed point theory of nonexpansive mappings
- Convexity and fixed point theory
- Properties of nonexpansive mappings and their use in practical applications
- Design of mappings having a desired fixed point set
Overview of fixed point algorithms for nonexpansive and quasi-nonexpansive mappings
- Mann algorithm (POCS will be stated here as an example),
- Haugazeau's algorithm and generalization
- Modern algorithms: Anchor method, Hybrid Steepest Descent method
Adaptive filtering paradigm revisited: advanced concepts
- Monotone approximation property of NLMS, APA
- Unified view of Adaptive Projected Subgradient Method
Applications to convexly constrained inverse problems
- Generalization of the Projected Landweber method
- Nonsmooth constrained problems
Applications
- Interference suppression in CDMA systems
- Advanced concepts in acoustic echo cancellation
- Other machine learning applications

Targeted Audience

The tutorial will be self-contained and will enable researchers and engineers to apply the concept immediately to their own problems. The material in the tutorial is suitable for generally knowledgeable audience, with basic knowledge of linear algebra, adaptive filtering and spectrum estimation.

Speaker Biographies

Dr. Mandic (www.commsp.ee.ic.ac.uk/~mandic) received the Ph.D. degree in nonlinear adaptive signal processing in 1999 from Imperial College. He is a Reader with the Department of Electrical and Electronic Engineering, Imperial College London, U.K.

He has written over 150 publications on a variety of aspects of signal processing, and has been a Guest Professor at the Catholic University Leuven, Belgium, Westminster University UK, and at TUAT Japan, and Frontier Researcher at the Brain Science Institute RIKEN, Tokyo, Japan. Dr. Mandic has been a Member of the IEEE Signal Processing Society Technical Committee on Machine Learning for Signal Processing, Associate Editor for IEEE Transactions on Circuits and Systems II, and Associate Editor for International Journal of Mathematical Modeling and Algorithms. He has won awards for his papers and for the products coming from his collaboration with industry.

His work on the applications of fixed point theory in engineering has been widely published, and some of the concepts relevant to this tutorial can be found in his research monographs Recurrent Neural Networks for Prediction (with J. Chambers), Wiley 2001, and Complex Valued Nonlinear Adaptive Filters (with S. L. Goh), Wiley 2007.

Dr. Yamada (www.comm.ss.titech.ac.jp/~isao/index.html) obtained his PhD degree in nonlinear signal processing in 1990 from Tokyo Institute of Technology, Tokyo, Japan. He is now an Associate Professor with the Department of Communications and Integrated Systems, Tokyo Institute of Technology, Tokyo, Japan. He has written over 150 publications on multidimensional, adaptive and array signal processing, optimization theory, and nonlinear inverse problems. Dr Yamada has been a Visiting Associate Professor at Pennsylvania State University, and has received awards for his research papers. He received the IEICE Young Researcher award in 1992, the ICF research award in 2004, and the DoCoMo mobile science award in 2005. Dr Yamada has served as an Associate Editor for IEEE Transactions on Circuits and Systems I, and for the International Journal on Multidimensional Systems and Signal Processing.