Nice simple explanation of Shannon Entropy.
Information, Uncertainty and Shannon Entropy – The Math Introduction at Nonoscience
The phrase ‘Information Entropy’ is one of the most confusing in science, since entropy is the lack of infomation. But the problem is not with the idea of equating information theory and entropy, just eth sloppy phrasing. Information Entropy means Entropy within the concept of information science (as opposed to thermodynamics, for example).