Some model are misclasified, appeared twice, etc. This page is a general map where you can find model types to learn.

Machine Learning Deep learning Deep cloning: way of doing online NLP, looking at relationship between words/ sentences NLP

  • Clustering
  • Anomaly detection
  • Regression
  • Classification
  • Recommander system

  • Information retrieval

Neural network

  • Perceptron
  • NN: Feed Forward Neural Network
  • CNN: Convolutional Neural Network
  • AN: Adversarial network
  • GAN: Generative Adversarial Networks
  • GSN: Generative Stochastic Networks (ie markov chain)
  • DCGAN: Deep convolutional GAN
  • LAPGAN: Laplacian Pyramidal GAN
  • Unrolled GAN (avoid collapse problem)
  • iGAN: interactive GAN. Draw something + give an image, and the network will regenerate it iGan
  • IAN: introspective AN: Apply modification to images by adding "color" which will be applied everywhere ... IAN
  • PPGN: Plug and Play Generative Networks
  • AE: Auto-Encoder
  • VAE: Variational Auto-Encoder
  • RBM: Restricted Boltzman Machine (symmetrically connected with hidden ynit)
  • DBN: Deep Belief Networks (composition of RBM / AE) Hopfield network (symmetrically connected without hidden unit)
  • FVBN: Fully visible belief networks

  • RNN: Recurrent network (need input ordonated with time)
  • LSTM: Long Short Term Mermory
  • ESN: Echo State Networks
  • Wave Net: Generative network for raw audio DeepMind (Network is organised as a binary tree) ArXiv

Graphical model

  • MC: Markov Chain
  • HMM: Hidden Markov Model
  • InfMC: Infinite Markov Chain
  • MDP: Markov Decision Process
  • MCMC: Markov Chain Monte Carlo
  • MCST: MonteCarlo Search Tree
  • CRF: Conditional Random Field
  • MAB: Multi armed bandit
  • Dynamical Neural Field

Way of learning from data

Abstract categories

  • Supervised Learning: Data + label/classification/...
  • Semi Supervised Learning: Some data are missing
  • Apprenticeship learning
  • Unsupervised Learning: No label for training (but you can have for testing and evaluating the model)
  • RL: Reinforcement Learning
    • Model based: with generative model
    • Model free: without
  • IRL: Inverse Reinforcement learning

Model capability

  • Generative
  • Discriminative

Model assumption

  • Stochastic
  • Deterministic

Data flow

  • Online Learning
  • Offline / Batch Learning
  • Mini batch
  • Stochastic

Measuring probability

  • MaP: Maximum a Posteriori
  • MLE: Maximum Likelihood Estimate
  • NCE: Noise Contrast Estimate
  • score matching
  • contrastive divergence


  • PCA: Principal Component Analysis
  • ICA: Independant Component Analysis TODO
  • LSA: Latent Semantic Analysis
  • Real NVP: real-valued non-volume preserving

non linear ICA: makes a non linear transformation from one space to another (like inverse function)

  • GD: Gradient descent
  • SGD: Stochastic Gradient Descent
  • ADAM ArXiv

  • Logistic regression

DataSplit and sampling

  • Holdout
  • k-fold
  • Leave-one-out
  • Jack-knife: Bias reduction by reestimating parameters when removing one of the variables.(oldschool)
  • Gibbs sampling
  • MCMC
  • Metropolis Hastings
  • Bootstraping (parametric and non-parametric)


  • SVM: Support Vector Machine
  • K-means
  • t-SNE


  • TF-IDF: Time Frequency - Inverse Document Frequency
  • n gram
  • BOW: Bag of Words
  • CBOW: Continuous BOW
  • skip gram
  • CSG: Continuous Skip Gram
  • LSA: Latent Semantic Analysis


ivan farkas