The Shannon's theorem reference article from the English Wikipedia on 24-Jul-2004
(provided by Fixed Reference: snapshots of Wikipedia from wikipedia.org)

Shannon's theorem

Sponsor with the world's largest charity for orphans
Shannon's theorem, proved by Claude Shannon in 1948, describes the maximum possible efficiency of error-correcting methods versus levels of noise interference and data corruption. The theory doesn't describe how to construct the error-correcting method, it only tells us how good the best possible method can be. Shannon's theorem has wide-ranging applications in both communications and data storage applications. This theorem is the foundation of the modern field of information theory.

In the communication domain, Shannon's theorem is known as the Shannon limit or Shannon capacity, the maximum rate of clean data C that can be sent through an analog communication channel subject to Gaussian-distribution noise interference:

where
C is the post-correction effective channel capacity in bits per second;
W is the raw channel capacity in hertz (the bandwidth); and
S/N is the signal-to-noise ratio of the communication signal to the Gaussian noise interference expressed as a straight power ratio (not as decibels)

Simple schemes such as "send the message 3 times and use a best 2 out of 3 voting scheme if the copies differ" are inefficient users of bandwidth and thus are far from the Shannon limit. Advanced techniques such as Reed-Solomon codes and, more recently, Turbo codes come much closer to reaching the theoretical Shannon limit, but at a cost of high computational complexity. With Turbo codes and the computing power in today's digital signal processors, it is now possible to reach within one-tenth of one decibel of the Shannon limit.

The V.34 modem standard advertises a rate of 33.6 kbit/s, and V.90 claims a rate of 56 kbit/s, apparently in excess of the Shannon limit. In fact, neither standard actually reaches the Shannon limit, but closely approaches it. The speed improvement of V.90 was made possible by the elimination of an additional step of analog to digital conversion by the use of fully digital equipment at the other end of a modem connection. This improves the S/N ratio, which in turn produces the required headroom to exceed 33.6 Kbps which was otherwise near the Shannon limit.

Table of contents
1 Examples
2 See Also
3 Reference

Examples

  1. If the SNR is 20 dB, and the bandwidth available is 4 kHz, which is appropriate for telephone communications, then C = 4 log 2 (1 + 100) = 4 log 2 (101) = 26.63 kbit/s. Note that the value of 100 is appropriate for an SNR of 20 dB.
  2. If it is required to transmit at 50 kbit/s, and a bandwidth of 1 MHz is used, then the minimum SNR required is given by 50 = 1000 log 2(1+S/N) so S/N = 2C/W -1 = 0.035 corresponding to an SNR of -14.5 dB. This shows that it is possible to transmit using signals which are actually much weaker than the background noise level, as in spread-spectrum communications.

See Also

Reference