The purpose of this study guide is to provide an introductory list of machine learning topics that you should know if you’re trying to become a Machine Learning practitioner. The guide is IN PROGRESS and I will improve it over time. I will also make short, informative YouTube videos about each topic to serve as jumping off points for further studying.

As always, please reach out if you have any thoughts or ideas that you would like to share! You can comment on the YouTube videos, reach out directly to me, or join the discord channel!

Machine Learning Basics (Introduction)

The content in this section should help provide a high level framework of machine learning. You can build upon this foundation as you learn more.

What is Machine Learning?

End-To-End Machine Learning Steps: EDA → Data Preprocessing → Training/Learning → Evaluation → Productionization

Math for Machine Learning

Math is the core of machine learning. To apply basic machine learning you don’t need to understand much math. To understand machine learning and apply it well you need to understand a decent amount of math. To become an expert in machine learning you need to understand math very well. This section goes over some of the most important concepts you need to understand machine learning.

Calculus

• Derivative
• Partial Derivatives / Gradient
• Chain Rule

Linear Algebra

• Vector
• Linear Combination
• Matrix
• Matrix Multiplication
• Dot Product

Statistics

• Probability
• Bayes Theorem
• Normal Distribution
• Skewed Data

Information Theory

• Information Gain
• Entropy

Dataset Basics

Machine learning is essentially programming a computer to learn from data. Therefore, this section goes over the basics of datasets.

• Structured vs Unstructured Data
• Example
• Label / target
• Feature
• Feature Vector
• Exploratory Data Analysis (EDA)

Data Preprocessing

This section covers what is needed to prepare your data so that you can perform machine learning with it.

• Data Preprocessing
• Data Cleaning
• Data Imputation
• Feature Scaling
  • Feature Normalization (Min-Max Scaling)
  • Feature Standardization (Mean Normalization; Z-Score Normalization)
• One Hot Encoding | Label Encoding
• Feature Engineering
• Binning (Discretization; Bucketing)
• Embedding
• Feature Selection
• Skewed Classes
• Data Augmentation
• Bootstrap / Bootstrapping
• Train Set vs Validation Set vs Test Set

Types of Machine Learning Algorithms

This section covers the different types of algorithms. It’s beneficial to have an understanding of what the types of algorithms are before you dive into how they are trained (learn).

• Model (Statistical Model)
• Instance Based Learning vs Model Based Learning
• Parametric vs Non-Parametric Learning Algorithms
• Supervised vs Unsupervised Learning Algorithms
  • Supervised
    • Regression vs Classification
    • Regression
      • Univariate Regression
      • Multiple Regression
      • Multivariate Regression
    • Classification
      • Binary Classification
      • Multiclass Classification (Multinomial Classification)
      • Multilabel Classification
      • Multioutput-Multiclass Classification (Multioutput Classification)
  • Unsupervised
    • Clustering
    • Anomaly Detection
• Dimensionality Reduction
• Reinforcement Learning
• Deep Learning (Neural Networks)
• Ensemble Learning
  • Boosting
  • Bagging (Bootstrap Aggregating)
• Generalized Linear Models

Training / Learning / Evaluating

Training (or learning) can be thought of as when a machine learning algorithm improves its ability to perform a task as it is continually evaluated. You evaluate the algorithm while it’s learning and you evaluate it after it has finished learning. Training and evaluating are interrelated. Therefore, the important concepts of each are covered in this section.

• Model Training
• Parameters vs Hyperparameters
• Hyperparameter Tuning (Hyperparameter Optimization)
• Gradient Descent
  • Batch Gradient Descent
  • Mini-Batch Gradient Descent
  • Stochastic Gradient Descent
  • Momentum (Gradient Descent with Momentum)
• Objective Function / Optimization Criterion
• Loss Function vs Cost Function
  • L1 Loss and Mean Absolute Error (MAE) (Average Absolute Error)
  • L2 Loss (Squared Loss) and Mean Square Error (MSE) (Square Error; Squared Error Loss)
• Training Error vs Generalization Error (Out-of-Sample Error)
• Holdout Validation
• Cross Validation
• Overfitting vs Underfitting
• Model Complexity
• Irreducible Error
• Bias vs Variance Tradeoff
• Learning Curve
• Regularization
  • l_k norm
  • L1 Regularization (Lasso Regularization)
  • L2 Regularization (Ridge Regularization)
  • Batch Regularization
  • Dropout Regularization (Dropout Normalization)
• Accuracy
• Confusion Matrix
• Precision
• Recall
• F1-Score
• AUC
• Manual Error Analysis
• Model Assessment
• Model Selection
• Interpretability and Explainability
• Training Time
• Transfer Learning and Fine Tuning
• Vanishing and Exploding Gradients

The Algorithms

The algorithms have been divided into classical algorithms (aka not neural networks) and deep learning algorithms (aka ones that use neural networks). This is how much of the literature is organized too.

Classical Algorithms

Supervised Machine Learning

• Linear Regression Algorithm
• Logistic Regression Algorithm
• Support Vector Machine Algorithm
• K-Nearest Neighbors Algorithm
• Decision Tree Algorithm
• Random Forrest Algorithm
• XGBoost Algorithm
• Naive Bayes Algorithm

Unsupervised Machine Learning

• K-Means Algorithm
• Principal Component Analysis
• Hierarchical Clustering Algorithm
• DBSCAN
• Gaussian Mixture Model (GMM)

Deep Learning (Neural Networks)

Perceptron

Activation Functions

• Linear Activation Function (Identity Activation Function)
• Logistic Function (Sigmoid Function)
• ReLU (Rectified Linear Unit Function)

Training

• Backpropagation
• Forwardpropagation

Deep Learning Architectures

Feed Forward Neural Network

Convolutional Neural Network

• Volume
• Convolution Operation
• Padding
• Pooling
• ResNet

Recurrent Neural Network

• Sequence Models
• Gated Recurrent Unit (GRU)
• Long Short-Term Memory (LSTM) Networks

Generative Adversarial Networks

Transformer Network

• Self-Attention (Scaled Dot-Product Attention)
• Large Language Models (LLMs)

Applications of Machine Learning

Natural Language Processing

• Dictionary (Vocabulary)
• Tokenization
• Stemming
• Lemmentization
• Bag-of-Words
• N-grams
• Word Embeddings
• TF-IDF
• Stop Words

Computer Vision

• Image Classification
• Object Detection
  • Sliding Window
• Semantic Segmentation

System Design and MLOps

This section outlines the important concepts for designing machine learning systems and running them in production environments.

• Machine Learning Pipeline
• Online Learning vs Offline Learning
• MLOps
• Model Decay (Model Deterioration; Model Rot; Model Drift)
• Data Distribution Shift