NOISE AND SIGNAL (RUIDO Y SEÑAL: ZEROHEDGE AND CLAUDE SHANNON)

Lot of noise out there lately. Here again is the signal pic.twitter.com/WUo5T5crGo

— zerohedge (@zerohedge) November 10, 2022

Noisy channel coding theorem and capacity[edit]

Main article: Noisy-channel coding theorem

Claude Shannon's development of information theory during World War II provided the next big step in understanding how much information could be reliably communicated through noisy channels. Building on Hartley's foundation, Shannon's noisy channel coding theorem (1948) describes the maximum possible efficiency of error-correcting methods versus levels of noise interference and data corruption.^[6][7] The proof of the theorem shows that a randomly constructed error-correcting code is essentially as good as the best possible code; the theorem is proved through the statistics of such random codes.

Shannon's theorem shows how to compute a channel capacity from a statistical description of a channel, and establishes that given a noisy channel with capacity ${\displaystyle C}$ and information transmitted at a line rate ${\displaystyle R}$, then if

${\displaystyle R<C}$

there exists a coding technique which allows the probability of error at the receiver to be made arbitrarily small. This means that theoretically, it is possible to transmit information nearly without error up to nearly a limit of ${\displaystyle C}$ bits per second.

The converse is also important. If

${\displaystyle R>C}$

the probability of error at the receiver increases without bound as the rate is increased. So no useful information can be transmitted beyond the channel capacity. The theorem does not address the rare situation in which rate and capacity are equal.