## Awesome Python

1. “and” and “or” returns the object >>> [] and {} [] # what??? Python’s “and” operation and “or” operation doesn’t returns “True” or “False”. A or B # is equal to A if A is True else B # and also A and B # is equal to A if A is False else B Note that neither and nor or restrict the value and type they return to False and True,...

## Entropy, Cross-Entropy, KL-Divergence

Entropy (at Information theory) The expectation of bits that used for notating (or classify each other) probabilistic events when using optimal bits coding scheme. ($$log_2(\frac{1}{p})$$ bits for notating events) Entropy also can be interpreted as the average rate at which information is produced by stochastic source of data. (rare events...

In Math If $p$ is probability.. odds is $\frac{p}{1-p}$. The logit (logistic unit) function or the log-odds is $logit(p) = \log \frac{p}{1-p}$ in statistics. Logit function makes a map of probability values from $(0, 1)$ to $(-\infty, +\infty)$. The logistic function or the sigmoid function is the inverse-logit. (\$log...