Shannon, information contained in a message in a noiseless channel:

N items (messages, characters)
Convert N to appropriate base (2 is most efficient)
Total information contained is log N i.e. bit count.
Information contained in one message is 1/N of total i.e. 1/N log N
log N = -log (1/N)
1/N is the probability of a particular message.
Therefore one char/message entropy is = -p log p
Total, H(X) = SUM -p log p