<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [ ['$','$'], ["\\(","\\)"] ],
displayMath: [ ['$$','$$'], ["\\[","\\]"] ],
processEscapes: true
},
"HTML-CSS": { availableFonts: ["TeX"] }
});
</script>
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.2/MathJax.js?config=TeX-AMS-MML_HTMLorMML" type="text/javascript">
</script>
entropic mind
Behnam Mohammadi Ph.D. Student, Marketing Tepper School of Business Carnegie Mellon University
Apr 6, 2020
Add $\LaTeX$ Functionality to Blogger
Apr 5, 2020
Information theory
If there’s a chance $p$ that your message gets flipped by the time it’s received at the other end of the line, is there any hope to transmit information in a way that on average, the probability of receiving the true message approaches 1?
· Reliable communication: communication with probability of error that can be made arbitrarily small.
One way is to add redundancy:
You can repeat yourself (add redundancy) to make the transmission more reliable, but at the same time, the rate of transmission decreases. For example, if you send each bit 3 times, the rate of transmission drops to 1/3 bit per channel use. If adding more and more redundancy increases the reliability, one might think that an infinite number of redundancies is needed to achieve true reliability. In that case, is there hope for reliable communication at all?! Can we increase reliability without forcing the rate to 0? According to Shannon, for each channel, there is in fact a range of rates that’s achievable with arbitrary reliability – as high reliability as you want; as close to probability 1 as you want. The maximum of such rates is Channel Capacity.
Now I understand how Shannon's theory of communication is counter-intuitive: you thought it's impossible to transmit any message reliably, but Shannon shows that you can pick any reliability (close to 1) and still be able to transmit information at a range of rates. This whole paradox reminds me of Zeno's paradox in which you think it's impossible to cross the street, but turns out you actually can.
Apr 4, 2020
Introducing the #LearntThisToday Posts
- By keeping track of what I learn everyday, I find myself more motivated to actually learn something. If a day goes by and I don't have anything to share in this category, then it means I should do better the next day. So, these posts are some kind of academic/scientific/fun diary of mine.
- Equally importantly, by writing down what I've learnt, I move the material from my short-term to long-term memory.
- And finally, these posts help me translate the more technical stuff into layman's terms. This helps me build a deep intuition about what I have learnt. I truly believe that the only time you can really claim you've learnt something is if you can explain it to your grandmother.