Operate ML models in production.
2 topics
MLOps applies DevOps practices to machine learning. The ML lifecycle: data collection → feature engineering → model training → evaluation → deployment → monitoring → retraining. Version everything: code (git), data (DVC, LakeFS), models (MLflow Model Registry), and experiments (W&B, MLflow). Reproducibility is the core challenge.
2 resources
Orchestrate training pipelines: data ingestion → preprocessing → training → evaluation → model registration. Tools: Kubeflow Pipelines, Vertex AI Pipelines, SageMaker Pipelines, ZenML. Pipelines should be: parameterized, reproducible, cached (skip unchanged steps), and triggered by data/schedule/manual. Feature stores (Feast, Tecton) ensure consistent features between training and serving.
Serve models as: REST APIs (FastAPI + custom), dedicated servers (TorchServe, Triton Inference Server, TF Serving), serverless (AWS Lambda, Google Cloud Functions), or edge (ONNX Runtime, TensorFlow Lite). Deployment strategies: shadow mode (compare new vs old), canary (gradual rollout), A/B testing. Optimize: batching, model quantization (FP16/INT8), caching, and GPU sharing.
Monitor: data drift (input distribution changes), concept drift (relationship between features and target changes), prediction quality, latency, and throughput. Tools: Evidently AI, WhyLabs, Arize. Set up automated alerts and retraining triggers. Implement feedback loops: collect ground truth, evaluate model performance continuously, retrain when metrics degrade.
1 topic
GPU management: NVIDIA drivers, CUDA, Docker with GPU support, Kubernetes GPU scheduling. Cloud ML platforms: AWS SageMaker, Google Vertex AI, Azure ML. Containerize training and serving with Docker. Use Kubernetes for orchestration — schedule GPU workloads, manage model servers, auto-scale inference. Cost optimization: spot/preemptible instances for training, right-size GPU instances, and model distillation to reduce serving costs.