Consider arbitrarily distributed input signals observed in additive Gaussian noise. A new fundamental relationship is found between the input-output mutual information and the minimum mean-square error (MMSE) achievable by any estimator of the input. That is, the derivative of the mutual information with respect to the signal-to-noise ratio is equal to half the MMSE. This identity holds for both scalar and vector signals, as well as for discrete- and continuous-time noncausal MMSE estimation (smoothing). The result uncovers intimate connections between information theory and estimation that can be exploited in many applications. A consequence of this result is a new relationship in continuous-time nonlinear filtering: Regardless of the input statistics, the optimal filtering MMSE and optimal smoothing MMSE determine each other.

Dongning Guo received the B.E. degree from the University of Science and Technology of China in 1995, the M.Eng. degree from the National University of Singapore in 1999, and the M.A. degree from Princeton University in 2001, all in electrical engineering. From 1998 to 1999, he was an R&D Engineer in the Centre for Wireless Communications, Singapore. He is currently a research assistant and Ph.D. candidate at the Department of Electrical Engineering, Princeton University. He will start a career as an assistant professor in a research university in Fall of 2004. His research interests are in communications and information theory. He received a Best Student Paper Award in the International Zurich Seminar on Broadband Communications in 2000.