AlgoPlusAlgoPlus
← All modules

Machine Learning

From a single perceptron to transformers and modern LLMs. · 36 topics

ML Foundations
Linear Regression→
Fit a line by minimizing squared error with gradient descent.
Logistic Regression→
A sigmoid squashes a linear score into a class probability.
Gradient Descent→
Roll downhill on the loss surface to find the minimum.
k-Nearest Neighbors→
Classify by the majority vote of the closest points.
Naive Bayes→
Classify with Bayes' rule assuming features are independent.
Decision Tree→
Split the data on the most informative feature, recursively.
Random Forest→
Many decorrelated trees vote for a robust prediction.
Support Vector Machine→
Find the maximum-margin separating hyperplane.
K-Means Clustering→
Group points around k moving centroids.
PCA→
Project data onto the directions of greatest variance.
Neural Networks
Perceptron→
The original neuron — a learned linear threshold classifier.
Multilayer Perceptron→
Stacked layers of neurons learn non-linear functions.
Activation Functions→
Sigmoid, tanh, ReLU, and softmax add non-linearity.
Backpropagation→
The chain rule sends error gradients backward through the net.
Optimizers (SGD & Adam)→
How gradients become weight updates — momentum, Adam.
Regularization & Dropout→
Fight overfitting with L2, dropout, and early stopping.
Deep Learning
Convolutional Neural Network→
Filters slide over an image to detect features.
Recurrent Neural Network→
A hidden state carries context across a sequence.
LSTM & GRU→
Gated memory cells learn long-range dependencies.
Autoencoder→
Compress to a bottleneck, then reconstruct the input.
Generative Adversarial Network→
A generator and discriminator train against each other.
Word Embeddings→
Map words to vectors where meaning is geometry.
Attention & Transformers
Attention Mechanism→
Weight every input by how relevant it is right now.
Self-Attention & Multi-Head→
Tokens attend to each other in parallel heads.
Positional Encoding→
Inject order into a permutation-invariant model.
Transformer Architecture→
Attention + feed-forward blocks, the modern backbone.
Large Language Models
Tokenization (BPE)→
Split text into subword tokens via byte-pair encoding.
Language Modeling→
Predict the next token from the ones before it.
Sampling & Decoding→
Temperature, top-k, and top-p shape what the model says.
GPT (Decoder-Only LLM)→
Masked self-attention stacks predict text left-to-right.
Fine-tuning & RLHF→
Adapt a base model, then align it with human feedback.
Retrieval-Augmented Generation→
Fetch relevant documents, then condition the answer on them.
Diffusion Models→
Generate images by learning to denoise pure noise.
Reinforcement Learning
Markov Decision Process→
States, actions, rewards — the framework for RL.
Q-Learning→
Learn action values from trial-and-error to act optimally.
Policy Gradient→
Directly optimize the policy by following reward gradients.