Machine Learning Basics

You do not need to derive gradient descent to work usefully alongside ML. You do need the vocabulary, and you need to recognise the handful of mistakes that make a model look better than it is.

The split. Data is divided into training, validation, and test sets. You fit on training, tune on validation, and touch test once, at the end. Every time you look at the test set and change something, you leak a little information into your model and your reported number drifts away from reality.

Overfitting and underfitting. An overfit model has memorised the training data, including its noise — near-perfect training scores, poor performance on anything new. An underfit model is too simple to capture the real pattern and does badly on both. The gap between training and validation performance is the diagnostic.

Metrics, and why accuracy lies. If 99% of transactions are legitimate, a model that predicts "legitimate" every time is 99% accurate and completely useless. Precision asks: of the things we flagged, how many were real? Recall asks: of the real ones, how many did we catch? They trade off against each other, and F1 is their harmonic mean. Which one matters is a product decision — a cancer screen wants recall, a spam filter wants precision.

Data leakage is the most common way a model looks brilliant in development and fails in production: some feature encodes the answer. A "customer_called_support" flag predicting churn is really recording that they already churned. Anything computed after the moment of prediction is leakage.

Class imbalance distorts both training and evaluation. The responses are resampling, class weighting, or picking a metric that is not fooled — and always checking the confusion matrix rather than a single number.

Training data is the product. Model architecture is usually the least interesting lever available to you; data quality, labelling consistency, and feature choice dominate results.