Information Theory - A Mathematical Theory of Communication
Claude Shannon's extraordinary idea – that basic principles of binary or digital information can be related to fundamental physical laws – was instrumental in shaping our digital era.
Today, Shannon’s theory remains the guiding foundation for communication scientists and engineers in their ongoing quest for faster, more energy efficient, and more robust communication systems.
Our ability to transform information – phone calls, music, video, virtually everything – into digital bits of data and to transmit billions of them per second is founded upon the innovative work of Bell Labs and MIT mathematician, Claude Shannon (1916-2001). Shannon’s seminal 1948 paper, “A Mathematical Theory of Communication,” established a whole new discipline, Information Theory, by showing how Boolean algebra and basic thermodynamic principles could be applied to communications.
Shannon provided a mathematical theory for encoding information by applying a value to it – either 0 or 1. This formulation is commonly known as the basis for digital communications. Furthermore, he demonstrated that mathematics could be used to calculate the theoretical maximum amount of information carried by a communications system based upon the physical laws of thermodynamics.
The most valuable aspect of Shannon’s work may be his definition of “information” for communication networks. This made it possible to identify the critical relationships between various network elements.
For example, Information Theory helps explain the interactions among the power of a particular signal, the bandwidth or frequency range of the channel through which the signal travels, and the channel noise, which alters the signal on its way to its destination.
Shannon’s equations told engineers how much information could be transmitted over the channels of an ideal system. He also spelled out mathematically the principles of “data compression,” which explains what the end of this sentence demonstrates, that “only infrmatn esentil to understandn mst b tranmitd.” And he showed how we can use controlled error rates to ensure integrity as information is transmitted over noisy channels.
Using Information Theory to Combat Climate Change When Shannon published his theory in 1948, the largest communications cable in operation could carry up to 1,800 voice conversations. Twenty-five years later, the highest capacity cable was carrying 230,000 simultaneous conversations.
Today a single strand of optical fiber as thin as a human hair can carry more than 6.4 million conversations. As communication channels today rapidly approach the theoretical limits set by Shannon, it becomes ever more important that we find innovative new methods of communication and methods of organizing communication systems and networks within these limits.
Scientists and engineers worldwide continue to expand on Claude Shannon’s ideas by conducting research into communications systems from a fundamental mathematical perspective and to guide the design of new telecommunications technologies.
Theoretical limits built upon Shannon’s theory indicate that the network data traffic generated by network users today could be transported using as little as 1 milliwatt of power. That’s 25,000 times less than the 25 watts of energy estimated to be consumed by a network user today using state of the art equipment.
Today, Shannon’s theory remains the guiding foundation for communication scientists and engineers in their ongoing quest for faster, more energy efficient, and more robust communication systems.
Our ability to transform information – phone calls, music, video, virtually everything – into digital bits of data and to transmit billions of them per second is founded upon the innovative work of Bell Labs and MIT mathematician, Claude Shannon (1916-2001). Shannon’s seminal 1948 paper, “A Mathematical Theory of Communication,” established a whole new discipline, Information Theory, by showing how Boolean algebra and basic thermodynamic principles could be applied to communications.
Shannon provided a mathematical theory for encoding information by applying a value to it – either 0 or 1. This formulation is commonly known as the basis for digital communications. Furthermore, he demonstrated that mathematics could be used to calculate the theoretical maximum amount of information carried by a communications system based upon the physical laws of thermodynamics.
The most valuable aspect of Shannon’s work may be his definition of “information” for communication networks. This made it possible to identify the critical relationships between various network elements.
For example, Information Theory helps explain the interactions among the power of a particular signal, the bandwidth or frequency range of the channel through which the signal travels, and the channel noise, which alters the signal on its way to its destination.
Shannon’s equations told engineers how much information could be transmitted over the channels of an ideal system. He also spelled out mathematically the principles of “data compression,” which explains what the end of this sentence demonstrates, that “only infrmatn esentil to understandn mst b tranmitd.” And he showed how we can use controlled error rates to ensure integrity as information is transmitted over noisy channels.
Using Information Theory to Combat Climate Change When Shannon published his theory in 1948, the largest communications cable in operation could carry up to 1,800 voice conversations. Twenty-five years later, the highest capacity cable was carrying 230,000 simultaneous conversations.
Today a single strand of optical fiber as thin as a human hair can carry more than 6.4 million conversations. As communication channels today rapidly approach the theoretical limits set by Shannon, it becomes ever more important that we find innovative new methods of communication and methods of organizing communication systems and networks within these limits.
Scientists and engineers worldwide continue to expand on Claude Shannon’s ideas by conducting research into communications systems from a fundamental mathematical perspective and to guide the design of new telecommunications technologies.
Theoretical limits built upon Shannon’s theory indicate that the network data traffic generated by network users today could be transported using as little as 1 milliwatt of power. That’s 25,000 times less than the 25 watts of energy estimated to be consumed by a network user today using state of the art equipment.