Discrete Stochastic Processes

Discrete Stochastic Processes

Follow Discrete Stochastic Processes
Share on
Copy link to clipboard

Lecture videos from 6.262 Discrete Stochastic Processes, Spring 2011. License: Creative Commons BY-NC-SA More information at ocw.mit.edu/terms

Robert Gallager


    • Jun 22, 2015 LATEST EPISODE
    • infrequent NEW EPISODES
    • 1h 19m AVG DURATION
    • 25 EPISODES


    Search for episodes from Discrete Stochastic Processes with a specific topic:

    Latest episodes from Discrete Stochastic Processes

    Lecture 25: Putting It All Together

    Play Episode Listen Later Jun 22, 2015 81:26


    In this lecture, we put together many of the topics covered throughout the term: martingales; Markov chains; countable state Markov processes; reversibility for Markov processes; random walks; and Wald's identity for two thresholds.

    Lecture 21: Hypothesis Testing and Random Walks

    Play Episode Listen Later Jun 22, 2015 85:22


    Sequential hypothesis testing is viewed as a random walk example. Threshold hypothesis tests are distinguished from random walk thresholds. Random walk threshold probabilities are analyzed by Chernoff bounds.

    Lecture 24: Martingales: Stopping and Converging

    Play Episode Listen Later Jun 22, 2015 80:43


    This lecture continues our conversation on Martingales and covers stopped martingales, Kolmogorov submartingale inequality, martingale convergence theorem, and more.

    Lecture 23: Martingales (Plain, Sub, and Super)

    Play Episode Listen Later Jun 22, 2015 82:39


    After reviewing Wald's identity, we introduce martingales and show they include many processes already studied. Next, submartingales, supermartingales, and stopped (simple, sub, super) martingales are introduced.

    Lecture 22: Random Walks and Thresholds

    Play Episode Listen Later Jun 22, 2015 81:17


    This lecture covers topics including the Kingman bound for G/G/1, large deviations for hypothesis tests, sequential detection, and tilted random variables and proof of Wald's identity.

    Lecture 20: Markov Processes and Random Walks

    Play Episode Listen Later Jun 22, 2015 83:08


    After reviewing steady-state, this lecture discusses reversibility for Markov processes and for tandem M/M/1 queues. Random walks and their applications are then introduced.

    Lecture 19: Countable-state Markov Processes

    Play Episode Listen Later Jun 22, 2015 82:14


    Markov processes with countable state-spaces are developed in terms of the embedded Markov chain. The steady-state process probabilities and the steady-state transition probabilities are treated.

    Lecture 18: Countable-state Markov Chains and Processes

    Play Episode Listen Later Jun 22, 2015 76:29


    In this lecture, the professor covers sample-time M/M/1 queue, Burke’s theorem, branching processes, and Markov processes with countable state spaces.

    Lecture 14: Review

    Play Episode Listen Later Jun 22, 2015 79:19


    This lecture reviews the previous 13 lectures in preparation for the upcoming quiz.

    Lecture 13: Little, M/G/1, Ensemble Averages

    Play Episode Listen Later Jun 22, 2015 74:52


    This lecture covers a variety of topics, including elementary renewal theorem, generalized stopping trials, the G/G/1 queue, Little's theorem, ensemble averages and more.

    Lecture 16: Renewals and Countable-state Markov

    Play Episode Listen Later Jun 22, 2015 79:40


    After reviewing the three major renewal theorems, we introduce Markov chains with countable state spaces. The matrix approach for finite-state chains is replaced by renewals based on first-passage times.

    Lecture 15: The Last Renewal

    Play Episode Listen Later Jun 22, 2015 75:43


    In this lecture, we continue our discussion of renewals and cover topics such as Markov chains and renewal processes, expected number of renewals, elementary renewal and Blackwell theorems, and delayed renewal processes.

    Lecture 17: Countable-state Markov Chains

    Play Episode Listen Later Jun 22, 2015 83:45


    This lecture begins with a discussion of convergence WP1 related to a quiz problem. Then positive and null recurrence, steady state, birth-death chains, and reversibility are covered.

    Lecture 11: Renewals: Strong Law and Rewards

    Play Episode Listen Later Jun 22, 2015 78:16


    This lecture begins with the SLLN and the central limit theorem for renewal processes. This is followed by the time-average behavior of reward functions such as residual life.

    Lecture 9: Markov Rewards and Dynamic Programming

    Play Episode Listen Later Jun 22, 2015 83:36


    This lecture covers rewards for Markov chains, expected first passage time, and aggregate rewards with a final reward. The professor then moves on to discuss dynamic programming and the dynamic programming algorithm.

    Lecture 8: Markov Eigenvalues and Eigenvectors

    Play Episode Listen Later Jun 22, 2015 83:37


    This lecture covers eigenvalues and eigenvectors of the transition matrix and the steady-state vector of Markov chains. It also includes an analysis of a 2-state Markov chain and a discussion of the Jordan form.

    Lecture 6: From Poisson to Markov

    Play Episode Listen Later Jun 22, 2015 79:16


    This lecture treats joint conditional densities for Poisson processes and then defines finite-state Markov chains. Recurrent and transient states, periodic states, and ergodic chains are discussed. (Courtesy of Mina Karzand. Used with permission.)

    Lecture 5: Poisson Combining and Splitting

    Play Episode Listen Later Jun 22, 2015 84:31


    In this lecture, many problem solving techniques are developed using, first, combining and splitting of various Poisson processes, and, second, conditioning on the number of arrivals in an interval.

    Lecture 12: Renewal Rewards, Stopping Trials, and Wald's Inequality

    Play Episode Listen Later Jun 22, 2015 86:21


    In this lecture, we learn about time-averages for renewal rewards, stopping trials for stochastic processes, and Wald's equality.

    Lecture 10: Renewals and the Strong Law of Large Numbers

    Play Episode Listen Later Jun 22, 2015 81:52


    Renewal processes are introduced and their importance in analyzing other processes is explained. Proofs about convergence with probability 1 (WP1) and the SLLN are given.

    Lecture 7: Finite-state Markov Chains; The Matrix Approach

    Play Episode Listen Later Jun 22, 2015 55:34


    The transition matrix approach to finite-state Markov chains is developed in this lecture. The powers of the transition matrix are analyzed to understand steady-state behavior. (Courtesy of Shan-Yuan Ho. Used with permission.)

    Lecture 3: Law of Large Numbers, Convergence

    Play Episode Listen Later Jun 22, 2015 81:27


    This lecture begins with the use of the WLLN in probabilistic modeling. Next the central limit theorem, the strong law of large numbers (SLLN), and convergence are discussed.

    Lecture 4: Poisson (the Perfect Arrival Process)

    Play Episode Listen Later Jun 22, 2015 77:13


    This lecture begins with a description of arrival processes, and continues on to describe the Poisson process from three different viewpoints.

    Lecture 2: More Review; The Bernoulli Process

    Play Episode Listen Later Jun 22, 2015 68:19


    The review of probability is continued with expectation, multiple random variables, and conditioning. We then move on to develop the weak law of large numbers (WLLN) and the Bernoulli process.

    Lecture 1: Introduction and Probability Review

    Play Episode Listen Later Jun 22, 2015 76:26


    Probability, as it appears in the real world, is related to axiomatic mathematical models. Events, independence, and random variables are reviewed, stressing both the axioms and intuition.

    Claim Discrete Stochastic Processes

    In order to claim this podcast we'll send an email to with a verification link. Simply click the link and you will be able to edit tags, request a refresh, and other features to take control of your podcast page!

    Claim Cancel