Further Topics in Probability 3

Unit aims

To outline, discuss, and prove with full mathematical rigour some of the key results in classical probability theory; with special emphasis on applications.

Unit description

This course deals with various analytic tools used and exploited in probability theory. Various modes of convergence of random variables (almost surely, weak, in probability, in Lp and in distribution) and the connections between them are presented. The key theorems are the Weak and Strong Laws of Large Numbers and the Central Limit Theorem. The analytic tools are: generating functions, Laplace- and Fourier transforms and fine analysis thereof.

Relation to other units

Measure Theory and Integration may be useful.

Learning objectives

To gain profound understanding of the basic notions and techniques of analytic methods in probability theory. In particular: generating functions, Laplace- and Fourier-transforms. To gain insight and familiarity with the various notions of convergence in the theory of random variables (in probability, almost sure, L^p, in distribution). Special emphasis will be on various “down-to-earth” applications of the mathematical theory.

Transferable skills

Model building, mathematically rigorous modeling of uncertainty. Connecting probabilistic ideas to analytic ones.


  1. The convolution: Discrete convolutions (on N and Z). BIN*BIN=BIN, POI*POI=POI, GEO*GEO=NEG_BIN. Absolutely continuous convolutions (on R): GAU*GAU=GAU, CAU*CAU=CAU stability. Convolutions of EXP: the Gamma-distributions. Relation to Poisson. Chi-square.
  2. Generating functions: Definition, examples. Reconstruction of the distribution from its generating function. Generating function of convolution, mixture of distributions, sums with random number of summands. Galton-Watson process – full analysis. Application to 1d random walks: hitting times, return times. Recurrence vs. transience. Weak convergence of discrete distributions with generating functions: Poisson approximation in more general context. 
  3. Weak law of large numbers and normal fluctuations: Markov’s and Chebyshev’s inequality and the WLLN with second moment. Application: Weierstrass’s Approximation Theorem – S. Bernstein’s proof. Fluctuations. Stirling’s formula and De Miovre’s theorem. Normal approximation of binomial, Poisson and gamma distributions
  4. Elements of measure theory: Sigma-algebra, measurability, generated algebra. Sigma-additive measure, extensions. Construction of Lebesgue measure. Continuity of the measure. Lebesgue integral, Monotone Convergence Theorem, Fatou’s Lemma, Dominated Convergence Theorem. Probability space – the general notion. Notions of convergence: almost sure, stochastic, L^p. Relations between these.
  5. Strong law of large numbers: Why “weak law”? Borel-Cantelli Lemmas. Strong Law of Large Numbers with fourth moment. Kolmogorov’s inequality. Kolmogorov’s Three Series Theorem. The Strong Law of Large Numbers in its full grandeur. (Kolmogorov) Kolmogorov’s 0-1 Law.
  6. Characteristic functions: Definition and basic properties. Computations for known distributions. Moments and derivatives of characteristic functions. The Moment Problem. Smoothness of the distribution and decay of the characteristic function. Decay of the distribution and smoothness of the characteristic function. Characteristic function of sums of independent random variables and forecast of the CLT. Inversion formula: Reconstruction of the distribution function from the characteristic function.  
  7. Weak convergence of probability distributions: The notion of weak convergence. Weak convergence as pointwise convergence of distribution functions. Example: DeMoivre’s theorem. Tightness: Helly’s Selection Theorem. Levy’s Lemma and limit theorem from pointwise convergence of characteristic functions.  The CLT in its full grandeur. (Markov + Levy).   Complementary comments to the CLT: local version, speed of convergence (Berry-Essen), Lindeberg’s theorem.
  8. Elements of large deviations (if time permits): The problem of large deviations for sums of i.i.d. random variables. Bernstein’s or Chernoff-Hoeffding inequalities. “Naïve” computations for some notable cases. Cramer’s Theorem.

Reading and References

  1. R. Durrett, Probability – Theory and Examples, Duxbury Press, 1995 
  2. W. Feller, An introduction to probability theory and its applications. Vols.1, 2, Wiley, 1970
  3. J. Lamperti, Probability -- a Survey of the Mathematical Theory, W.A Benjamin Inc., New York-Amsterdam, 1966
  4. S. Resnick, Adventures in Stochastic Processes, Birkhauser, 1992
  5. A. N. Shiryaev, Probability (Second Edition), Springer, Graduate Texts in Mathematics 95, 1996
  6. D. Williams, Probability with Martingales, Cambridge University Press, 1991

Instructor’s lecture notes and problem sheets.

Unit code: MATH30006
Level of study: H/6
Credit points: 20
Teaching block (weeks): 2 (13-24)
Lecturer: Dr Joseph Najnudel and Dr Marton Balazs


MATH20008 Probability 2



Methods of teaching

Lectures supported by problem sheets and solution sheets.

Methods of Assessment

The pass mark for this unit is 40.

The final mark is calculated as follows:

  • 80% from a 2 hour 30 minute exam in May/June
  • 20% assessed homework*

*homework will be assigned on a bi-weekly basis. An assignment will typically consist of about 10 problems; approximately half of the problems will be marked and count towards the final assessment mark.

NOTE: Calculators are NOT allowed in the examination.

Statistical tables will be provided.

For information resit arrangements, please see the re-sit page on the intranet.

Please use these links for further information on relative weighting and marking criteria.

Further exam information can be found on the Maths Intranet.

Edit this page