Properties Analysis of Smoothing Method for Mandarin Language Models

Abstract

In this paper, a set of statistical properties is used to evaluate several well-known smoothing methods. We first propose a set of properties to analyze the statistical behaviors of these methods. Furthermore, we present a new smoothing method which complies with all the proposed properties. Finally, we implement three Mandarin language models and then evaluate the cross entropies on various size N.