From regression to neural networks.
3 topics
Linear algebra is the mathematical backbone of machine learning. Vectors represent data points and features, matrices represent datasets and transformations. Key concepts: vector norms, matrix multiplication, eigendecomposition (PCA), singular value decomposition, and broadcasting. NumPy is your computational tool for all linear algebra.
2 resources
ML is an optimization problem — find parameters that minimize loss. Calculus tools: derivatives, partial derivatives, chain rule (backpropagation), and gradient descent variants (SGD, Adam). Understand learning rate schedules, convergence, and local vs global minima.
Statistics underpins model evaluation and data understanding. Master: probability distributions, Bayes theorem, hypothesis testing, confidence intervals, A/B testing, and the bias-variance tradeoff. These concepts apply to model evaluation, feature selection, and communicating uncertainty.
2 topics
Supervised learning maps inputs to outputs using labeled data. Regression predicts continuous values (linear, polynomial, ridge/lasso). Classification predicts categories (logistic regression, decision trees, random forests, SVMs, k-NN). The workflow: split data, train, evaluate with metrics (MSE/RMSE for regression, accuracy/precision/recall/F1/AUC-ROC for classification), tune hyperparameters, and interpret results.
Ensemble methods combine models for better performance. Bagging (Random Forest) trains on random subsets. Boosting (XGBoost, LightGBM, CatBoost) trains sequentially, each correcting previous errors. Gradient boosting is the best algorithm for tabular data — consistently beats deep learning on structured datasets. Key hyperparameters: number of trees, learning rate, max depth, and regularization.
Unsupervised learning finds patterns in unlabeled data. Clustering: k-means, DBSCAN, hierarchical, GMMs. Dimensionality reduction: PCA (linear), t-SNE and UMAP (non-linear visualization). Applications: customer segmentation, anomaly detection, feature engineering, and data exploration.
Neural networks learn complex patterns through layers of neurons. Key concepts: feedforward networks, activation functions (ReLU, sigmoid, softmax), loss functions, backpropagation, and optimizers (Adam). Regularization: dropout, batch normalization, early stopping. Learn PyTorch: nn.Module, DataLoaders, GPU acceleration, and model checkpoints. Start simple before moving to CNNs and Transformers.
Feature engineering transforms raw data into model inputs — often more important than algorithm choice. Techniques: handling missing values, encoding categoricals, scaling numericals, interaction features, time-based features, and domain-specific transforms. Use scikit-learn pipelines to prevent data leakage and ensure reproducibility. Feature stores (Feast) manage features for production.
Deploy models as REST APIs (FastAPI), use model servers (TorchServe, Triton), or serverless. Optimize with quantization, pruning, and ONNX Runtime. Monitor: prediction distributions (data drift), performance over time (concept drift), latency, and error rates. Retrain on schedule or when drift is detected. A/B test new models before full rollout.