Lecture Notes on statistics and information Theory

Overview & Synopsis

This
lecture note navigates through information theory, statistics and measure theory. It
covers fundamental concepts such as definitions, chain rules, data processing
inequalities, and divergences and extends to optimal procedures, LeCams and
Fanos inequalities, and operational results like entropy and source coding. It
also focus on exponential families and statistical modeling, fitting procedures,
and lower bounds on testing parameters, subGaussian and subexponential random
variables, martingale methods, uniformity covering topics such as
KullbackLeibler divergence, PACBayes bounds, interactive data analysis, and
error bounds.