Home | Blog | Wiki | About

• Transfer learning: 'storing knowledge gained while solving one problem and applying it to a different but related problem. For example, knowledge gained while learning to recognize cars could apply when trying to recognize trucks'

• Reinforcement learning:

  • 'focus is on finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).'
  • No need of labelled data or explicit correction. Sort of learns by itself based on what it sees.
  • 'concerned with how intelligent agents ought to take actions in an environment in order to maximize the notion of cumulative reward'

• ReLu: Rectified linear unit

  • A popular activation function in ANNs
  • f(x) = max(0, x)

• Meta-learning: 'Learning to learn'

  • https://lilianweng.github.io/posts/2018-11-30-meta-learning/

• Support vector machine (SVM)

  • For (linear) classification problem
  • Supervised training

• Perceptron (McCulloch-Pitts)

• Linear classification

  • Points of different classes drawn on a graph can be demarcated by straight lines

• Support vector machines (SVM): Can do linear classification

• Forecasting algorithms

  • ARIMA (Auto-regressive integrated moving average)
  • SARIMA (Seasonal auto-regressive integrated moving average)

• Autoregression

• Moving average

Reinforcement learning

• Value iteration

• Q-Learning

• Policy (π)

  • Are probabilities of 'transitions'
  • P(s'|s, a): probability of going to s' from s by taking action a
  • Expected utility of a π
  • Probabilistic => need not take the same path every time for the same π
  • π^*: optimal policy
    • Policy with maximum expected utility

• Discounted rewards

  • Discount factor = γ

• R(s): reward associated with state s

• Markov process

  • Next state is independent of history

• MDP: Markov decision process

—

Bellmann equation for utility

U(s) = R(s) + γ max [P(s'|s, a) * U(s')]
                a∈A

This essentially says this:

Utility of a state = Reward of that state + Utility of next state

Sketching

From here:

A sketch C(X) of some data set X with respect to some function f is a compression of X that allows us to compute, or approximately compute, f (X) given access only to C(X).

• 'compress data in a way that lets you answer queries' ¹¹
• Helps save bandwidth when streaming data over a network.