2019: PP: Sam Staton
Lecture 1
Bus weekend example
On an average:
- Weekend => 2 buses in an hour
- Weekday => 10 buses in an hour
Saw 4 buses.
Is it a weekend?
Let's consider the probability of it being a weekend.
x
Buses showing up.
- Same event happening multiple times within a time period.
- Matches description of a Poisson distribution.
Example languages:
- Hakaru: Indiana
- PSI: ETH Zürich
—
Methods of running a probabilistic program
- 'Direct calculation'
- 'Simulation method'
—
:Side-note:
- Definitional interpretor
- By John Reynolds
- Describing a programming language by defining an interpreter for it
- Focus is on meaning, not efficiency
- Dana Scott and Christopher Strachey proposed denotational semantics
- TODO: What's the difference from definitional interpreter?
- Denotational semantics: Describes what a program does by associating it with a mathematical object
- Definitional interpretor: Describes what a program does by giving an interpretor giving that mathematical object ??
Simulation
Let's see two ways:
- Monte Carlo with rejection
- Weighted Monte Carlo
- aka importance sampling Monte Carlo
Monte Carlo with rejection
- Run the program numerous (a big number) times
- Make random choice on each run
Note: Likelihood and probability are not the same. The subtle difference is more clearly in the case of continuous.
TODO: How does one decide which probability distribution would be best suited to a scenario?
Uncertainity
Incorporating uncertainity about facts into the model.
- Gamma distribution
- Rate/mean = value that we think it is
- Changing observation
Regression
Tries to answer this question: 'What function could have generated these data points?'
- Linear regression => this function is a straight line.
- 'Traditional statistics'=> find the line that fits best. Least squares method.
- Bayesian statistics
- TODO: Cubic functions will have 4 parameters ?? Why?
Metropolis-Hastings algorithm
Lecture 2
- An attraction of probabilistic programming: Model is separate from the inference algorithm