Maximum Entropy Techniques and Exponential Models in SLM/SCL

Wednesday, November 1, 2000 - 9:30am - 10:30am
Keller 3-180
Sanjeev Khudanpur (Johns Hopkins University)
Maximum Entropy methods provide elegant means for estimating complex probabilistic models from sparse data. They have been applied to various problems encountered in the processing of natural language including statistical language modeling, part-of-speech tagging, parsing, text segmentation and classification and machine translation. This presentation will review of some of these applications. It will be argued that the main issues in using maximum entropy techniques are (i) the selection of features or linear constraints which the model should be required to satisfy and (ii) the enormous computation involved in obtaining model parameters given these constraints. Recent proposals in the literature for addressing these two problems will be mentioned and some open problems will be brought up for discussion.