From statistics to deep learning research.
2 topics
Mathematics is the language of machine learning. Linear algebra underpins nearly every ML algorithm: vectors and matrices, matrix multiplication, eigenvalues and eigenvectors (used in PCA), singular value decomposition (used in recommendations), and vector spaces. Calculus provides the foundation for optimization: derivatives (how functions change), partial derivatives (gradients in multi-dimensional space), chain rule (backpropagation in neural networks), and gradient descent (iteratively minimizing loss functions). You do not need to prove theorems, but you must be able to reason about dimensions, understand what a gradient means intuitively, and know why certain operations work. NumPy makes all of this computational — learn to think in terms of array operations rather than for loops.
2 resources
Probability and statistics form the theoretical backbone of data science. Master: probability distributions (normal, binomial, Poisson, uniform), Bayes theorem (updating beliefs with evidence), maximum likelihood estimation, hypothesis testing (t-tests, chi-square, ANOVA), confidence intervals, p-values (and their limitations), correlation and regression, A/B testing with statistical power analysis, and Bayesian statistics (priors, posteriors, credible intervals). Understand the central limit theorem (why averages are normally distributed regardless of underlying distribution), law of large numbers, and how to handle multiple comparisons (Bonferroni correction). These concepts directly apply to model evaluation, feature selection, experiment design, and communicating uncertainty to stakeholders.
Master the core machine learning algorithms and when to use each. Supervised learning: linear regression (continuous targets), logistic regression (binary classification), decision trees (interpretable rules), random forests (ensemble of trees, robust), gradient boosting (XGBoost, LightGBM, CatBoost — best tabular performance), SVMs (support vector machines for high-dimensional data), and k-nearest neighbors. Unsupervised learning: k-means clustering, hierarchical clustering, DBSCAN (density-based), PCA (dimensionality reduction), and t-SNE/UMAP (visualization). The full ML workflow: data cleaning, exploratory data analysis, feature engineering (creating informative features from raw data), feature selection, model training, hyperparameter tuning (grid search, random search, Bayesian optimization), cross-validation, evaluation metrics, and model interpretation (SHAP, feature importance).
Deep learning uses multi-layer neural networks to learn complex patterns from large datasets. Foundations: perceptrons, activation functions (ReLU, sigmoid, softmax), loss functions (cross-entropy, MSE), backpropagation, and gradient descent variants (SGD, Adam, AdamW). Architectures: CNNs (convolutional neural networks for images — convolution layers, pooling, transfer learning with ResNet/EfficientNet), RNNs/LSTMs (sequence data — though largely replaced by Transformers), Transformers (attention mechanism — the foundation of modern AI), and GANs (generative adversarial networks for image generation). Learn PyTorch deeply — tensors, autograd, custom modules, DataLoaders, training loops, mixed precision training, and distributed training. Use pre-trained models from Hugging Face and fine-tune for your specific task. Experiment tracking with MLflow or Weights & Biases is essential.
NLP enables machines to understand, generate, and reason about human language. Traditional NLP: tokenization, stemming/lemmatization, TF-IDF, and word embeddings (Word2Vec, GloVe). Modern NLP is dominated by Transformers: BERT for understanding tasks (classification, NER, question answering), GPT for generation, and T5 for sequence-to-sequence tasks. Key tasks: text classification (sentiment, topic), named entity recognition (extracting people, places, organizations), question answering, summarization, translation, and semantic similarity. Fine-tune pre-trained models from Hugging Face for your specific domain. Understand evaluation metrics: BLEU (translation), ROUGE (summarization), perplexity (language models), and F1 (classification). The frontier is now LLM-based NLP — using prompting and RAG instead of traditional fine-tuning for many tasks.
Computer vision enables machines to interpret images and video. Core tasks: image classification (what is in the image?), object detection (where are objects and what are they? — YOLO, Faster R-CNN), semantic segmentation (pixel-level classification), instance segmentation (separate each object), and image generation (diffusion models, GANs). Transfer learning is the standard approach: take a model pre-trained on ImageNet (ResNet, EfficientNet, Vision Transformer) and fine-tune on your specific dataset. Data augmentation (rotation, flipping, color jittering, Mixup, CutMix) is critical when your dataset is small. Libraries: torchvision, Ultralytics (YOLO), OpenCV for image processing, and Hugging Face for vision transformers. Evaluation metrics: accuracy, mAP (mean Average Precision for detection), and IoU (Intersection over Union for segmentation).
Rigorous experimentation is what separates data science from ad hoc analysis. Design experiments with clear hypotheses, control variables, appropriate sample sizes (power analysis), and proper randomization. For ML experiments: always split data into train/validation/test sets, use cross-validation for reliable estimates, track every experiment (hyperparameters, data versions, results) with tools like MLflow or Weights & Biases, and report results with confidence intervals, not just point estimates. For A/B tests: calculate required sample size before starting, check for novelty effects, segment results by important dimensions, and use sequential testing methods for early stopping. Guard against common pitfalls: data leakage (test data leaking into training), p-hacking (running many tests until one is significant), and confirmation bias (only looking for evidence that supports your hypothesis).
Staying current with research is essential for a data scientist. Learn to efficiently read academic papers: start with the abstract and conclusion, then figures and tables, then methodology if the results are relevant. Key venues: NeurIPS, ICML, ICLR (machine learning), ACL, EMNLP (NLP), CVPR, ICCV (vision), and arXiv (preprints). Use tools like Semantic Scholar, Papers With Code, and Twitter/X to discover relevant work. For publishing: reproduce existing results first, identify a clear contribution, run thorough experiments with proper baselines, write clearly following the venue's format, and contribute to open-source implementations. Even if you do not pursue academia, reading papers helps you adopt state-of-the-art techniques and understand the reasoning behind tools you use daily.