🤖AI & Machine Learning

Top 25 Machine Learning
interview questions

Bias-variance, model selection, evaluation, features and MLOps. Open a question to read the answer.

01

Explain supervised versus unsupervised learning in the context of Machine Learning.

Supervised learning fits a function f(x) ≈ y from examples where the target y is known; unsupervised learning finds structure in x alone, with no target. The distinction is about what the training data contains, not about which algorithm family is used.

Between the two sit semi-supervised learning (a few labels, many unlabelled rows) and selfsupervised learning, where the labels are derived from the data itself, as in masked-token prediction for language models. Reinforcement learning is a third paradigm with rewards instead of labels.

In a real project using Machine Learning, how would you apply, test, or troubleshoot supervised versus unsupervised learning?

Apply. In a payments fraud system I would use both. Labelled chargebacks feed a supervised gradient-boosting classifier for known fraud patterns. Because labels arrive weeks late and new fraud schemes have no labels at all, an unsupervised layer (Isolation Forest or an autoencoder on transaction features) flags anomalous behaviour for analyst review. Cluster assignments from kmeans on merchant behaviour also become features for the supervised model.

Test. The supervised model is tested against a time-based hold-out with precision at a fixed alert budget. For the unsupervised layer there is no ground truth, so I test it two ways: inject synthetic anomalies and measure recall on them, and track how many analyst-reviewed alerts from the anomaly detector are later confirmed as fraud. I also assert cluster stability across weekly refits (adjusted Rand index above an agreed threshold).

02

Explain classification versus regression in the context of Machine Learning.

Classification predicts a discrete label from a finite set; regression predicts a continuous numeric quantity. Both are supervised, and many model families (trees, neural networks, linear models) have a variant for each; what changes is the output layer, the loss function and the evaluation metrics.

The boundary is softer than it looks. Logistic regression is a regression on log-odds used for classification. Ordinal targets (satisfaction 1–5) can be treated either way. A continuous target can be binned into classes when the business only needs bands, and a classifier’s probabilities are themselves a regression onto [0, 1]. Multi-label classification (several labels per row) and multi-output regression extend each side.

In a real project using Machine Learning, how would you apply, test, or troubleshoot classification versus regression?

Apply. Framing is the first design decision. For a delivery-time product I chose regression on minutes (quantile loss at the 50th and 90th percentiles so the app can show a range) rather than classifying into “under 30 / 30–60 / over 60” buckets, because the buckets would have thrown away information the pricing team also needed. For churn I kept binary classification but trained on calibrated probabilities so retention offers could be ranked by expected value, not by a hard label.

Test. For the regressor, unit tests check that MAE on the hold-out beats a seasonal-median baseline and that residuals are not biased by region or hour of day. For the classifier, I test the threshold explicitly: a test pins precision and recall at the chosen operating point, and a calibration test checks that predicted 0.3 means roughly 30% observed churn (Brier score and reliability bins). Both get a metamorphic test: scaling an irrelevant feature must not change predictions.

03

Explain train validation test splits in the context of Machine Learning.

A dataset is partitioned into three disjoint subsets with different jobs. The training set fits model parameters. The validation set is used repeatedly during development to choose hyperparameters, features and architectures, and to decide when to stop training. The test set is touched once, at the end, to produce an unbiased estimate of generalisation performance.

The reason for three rather than two is selection bias: every decision made by looking at validation scores leaks a little information about that set into the model, so the validation score becomes optimistically biased. The test set, never used for decisions, remains honest.

In a real project using Machine Learning, how would you apply, test, or troubleshoot train validation test splits?

Apply. On a demand-forecasting project for a retailer, the split is temporal: train on months 1–18, validate on months 19–21, test on months 22–24, and the split is created once by a versioned script whose output (row IDs per subset) is stored alongside the data. The test set lives in a separate storage prefix that the training pipeline has no read permission for, so nobody can accidentally tune against it. For the churn model, splitting is by customer ID with stratification on the label, because one customer contributes many monthly snapshots.

Test. Automated checks run on every split artefact: the three ID sets have empty pairwise intersections; label prevalence per subset is within one percentage point of the full data; for the temporal split, max(train date) < min(validation date); group integrity holds (no customer in two sets). A pipeline test also trains on the training set alone and asserts that the preprocessing object was fitted only on training rows by checking its learned statistics.

04

Explain cross validation strategies in the context of Machine Learning.

Cross-validation estimates generalisation error by repeatedly partitioning the training data into fit and evaluation folds, so every row is evaluated exactly once while the model never scores rows it was fitted on. The mean and standard deviation across folds give both an estimate and its uncertainty, which a single validation split cannot.

Any preprocessing that learns from data must be fitted inside each fold, which is why scikit-learn’s Pipeline combined with cross_val_score is the standard idiom.

In a real project using Machine Learning, how would you apply, test, or troubleshoot cross validation strategies?

Apply. For a credit-risk scorecard with 40,000 applicants and a 4% default rate, I use stratified 5-fold cross-validation wrapped around a Pipeline (imputer, encoder, scaler, model) so every learned transform is refitted per fold. Hyperparameter search runs inside the folds; the final generalisation estimate comes from a nested outer loop on the most promising two or three configurations only, since full nesting over the whole grid is too slow. For a sensor-failure model where each machine contributes thousands of readings, GroupKFold by machine ID is non-negotiable.

Test. A test asserts that fold-level scores have a coefficient of variation below an agreed limit; wildly different folds mean the split is wrong or the data is too small. Another test compares train_score with test_score per fold to catch overfitting early. For the grouped case, a unit test confirms no group ID appears in both the fit and evaluation indices of any fold.

05

Explain bias variance tradeoff in the context of Machine Learning.

Expected prediction error at a point decomposes into three parts: error = bias² + variance + irreducible noise. Bias is the systematic gap between the average model (over many training sets) and the true function; variance is how much the fitted model changes when the training sample changes; noise is the floor no model can beat.

Model capacity moves the two terms in opposite directions. A linear model on a curved relationship has high bias and low variance: refitting on new samples gives nearly the same wrong line. A deep, unpruned decision tree has low bias and high variance: it fits each sample’s quirks and changes shape with every refit. The total error is typically U-shaped in capacity, and the goal is the minimum, not zero bias.

In a real project using Machine Learning, how would you apply, test, or troubleshoot bias variance tradeoff?

Apply. On a pricing model I treat the tradeoff as a budget decision made with learning curves. I train on 10%, 20%, … 100% of the data and plot training and validation error. If the curves converge at a high error, more data will not help and I need more capacity or better features; if a large gap persists and the validation curve is still falling, collecting more data or regularising is the better investment than redesigning the model. This decides where a sprint goes.

Test. The CI job for every candidate model records training score, validation score and their gap. A test fails if the gap exceeds a threshold agreed per project (for the pricing model, validation RMSE more than 15% above training RMSE), and another fails if training error is worse than a trivial baseline, which signals bias. Fold-to-fold standard deviation from cross-validation is logged as a direct variance measurement.

06

Explain overfitting and regularization in the context of Machine Learning.

Overfitting is when a model learns patterns specific to the training sample, including noise, so training error keeps falling while validation error rises. Regularisation is any technique that constrains the model so that it prefers simpler solutions, trading a little training error for better generalisation.

The strength λ is a hyperparameter chosen by cross-validation; too much regularisation swings the model into underfitting. Scaling features first matters for L1 and L2 because the penalty is applied per coefficient and a feature measured in cents would be penalised differently from the same feature in dollars.

In a real project using Machine Learning, how would you apply, test, or troubleshoot overfitting and regularization?

Apply. On a text-classification service for support tickets with 30,000 labelled examples and a 200,000-dimensional TF-IDF space, overfitting is the default failure mode. I use logistic regression with elastic-net penalty, tune C and l1_ratio by cross-validation, and keep the sparse model that survives. For the gradient-boosted tabular model next to it, the regularisation knobs are a low learning rate (0.03), max_depth of 4–6, min_child_weight, column subsampling and early stopping on a validation fold with early_stopping_rounds=50.

Test. Training runs log the training-versus-validation loss curve as an artefact, and a test fails the build if validation loss has risen for more than the patience window without early stopping triggering, which would mean the callback is miswired. A second test retrains on a 50% subsample and asserts the validation score drops by less than a fixed amount; a model that collapses when given half the data is memorising. Weight-norm or tree-count metrics are tracked per release so a sudden jump is visible.

07

Explain feature engineering in the context of Machine Learning.

Feature engineering is the transformation of raw data into the numeric representation a model learns from, using domain knowledge to expose signal the algorithm would otherwise have to discover on its own or could never discover at all. It remains the largest lever on tabular-model performance, more than algorithm choice.

Two rules govern every feature: it must be computable at prediction time from data that exists then (no future information), and any statistic it learns must be fitted on training data only. Tree ensembles are indifferent to monotone transforms like log, whereas linear models and neural networks benefit greatly, so the engineering effort is model-dependent.

In a real project using Machine Learning, how would you apply, test, or troubleshoot feature engineering?

Apply. For a subscription churn model, raw data is an events table and a billing table. I build features in a feature store-style layer with point-in-time semantics: for each customer and snapshot date, compute 30/90-day login counts, trend (this month minus last month), days since last support ticket, payment failure count, and plan-change flags, using only events strictly before the snapshot. Every feature is defined once in code and shared between training and the online scoring service, so the two cannot drift apart.

Test. Feature code gets unit tests on tiny hand-built frames with known answers (a customer with three logins in the window must yield 3). A point-in-time test shifts the snapshot date and asserts features change only when events before it change. In CI, a schema and distribution test compares new feature batches with the previous week’s (null rate, mean, quantiles) and fails on large shifts. Training-serving skew is tested by scoring the same customers through both the batch and online paths and asserting identical feature vectors.

08

Explain missing value imputation in the context of Machine Learning.

observed columns (MAR), or missing not at random (MNAR), where the fact of missingness itself carries information, such as income left blank by high earners.

The imputer must be fitted on training data inside the pipeline and applied unchanged to validation, test and production data; recomputing the median on the test set is a leak.

In a real project using Machine Learning, how would you apply, test, or troubleshoot missing value imputation?

Apply. In a clinical-risk model built from electronic health records, lab values are missing for most patients because tests are ordered only when a doctor suspects a problem, which is textbook MNAR. I impute numerics with the training median, add a missing indicator for every lab column, and let the gradient-boosting model learn that “no troponin test ordered” is itself a strong negative signal. Categorical fields get an explicit "unknown" level. For the linear baseline used by the clinicians, I use IterativeImputer because mean imputation would destroy the correlations between labs it depends on.

Test. Pipeline tests confirm the imputer is fitted only on the training fold: the learned statistics_ are compared against medians computed by hand on the training frame. A data-quality test fails if any production feature’s null rate moves more than ten percentage points from the training baseline, because imputation tuned for 30% missing behaves differently at 80%. An ablation experiment with and without indicator columns is recorded so the choice is evidence-based.

09

Explain categorical encoding in the context of Machine Learning.

Categorical encoding converts discrete labels (country, product type, device model) into numbers a model can consume, and the choice of encoding changes what the model can learn. Nominal categories have no order; ordinal categories (small, medium, large) do, and should be encoded with integers that preserve it.

One-hot encoding with thousands of levels creates sparse, wide matrices and lets trees waste splits; target encoding compresses it to one informative column but leaks the label unless computed with cross-fitting and smoothing toward the global mean. Modern tree libraries (LightGBM, CatBoost, scikit-learn 1.4+ HistGradientBoosting with categorical_features) accept categorical columns natively and partition levels optimally, which often beats manual encoding.

In a real project using Machine Learning, how would you apply, test, or troubleshoot categorical encoding?

Apply. An ad click-through model has advertiser_id with 200,000 levels, country with 150 and device_class with 5. I one-hot the small ones, use out-of-fold target encoding with smoothing for advertiser_id in the gradient-boosting model, and feed the raw IDs as learned embeddings in the neural model. New advertisers map to the global prior. The encoder is a fitted object serialised with the model so the serving path uses exactly the same vocabulary and smoothing constants.

Test. A leakage test trains on target-encoded features and checks that a level occurring once does not receive its own label as the encoded value. A round-trip test asserts that a serialised encoder reproduces training-time columns bit for bit. An unseen-level test scores a row with country="XX" and expects a valid prediction rather than an exception.

10

Explain normalization versus standardization in the context of Machine Learning.

Both rescale numeric features so that no feature dominates purely because of its units, but they map to different ranges and respond differently to outliers. Normalisation (min-max scaling) maps each feature linearly to a fixed interval, usually [0, 1]: x’ = (x − min) / (max − min). Standardisation (zscoring) centres each feature on zero with unit variance: x’ = (x − μ) / σ.

Which models care: gradient descent converges faster on scaled features; L1/L2 regularisation penalises coefficients fairly only when features share a scale; distance-based methods (k-NN, kmeans, SVM with RBF kernel) and PCA are dominated by large-range features without scaling. Treebased models are invariant to monotone rescaling and do not need either.

In a real project using Machine Learning, how would you apply, test, or troubleshoot normalization versus standardization?

tuned for that range. The gradient-boosting churn model receives raw features; scaling it would be wasted work.

Test. A pipeline test asserts that the scaler’s mean_ and scale_ match statistics computed independently on the training frame, confirming it was not fitted on the full dataset. For the k-means pipeline, a test checks that the fraction of total variance contributed by each scaled feature is within an order of magnitude of the others. An inference test feeds a feature value 10x the training maximum and asserts the service neither crashes nor silently clips unless clipping is configured.

11

Explain class imbalance techniques in the context of Machine Learning.

Class imbalance occurs when one class vastly outnumbers another, such as 0.2% fraudulent transactions, so a classifier can achieve high accuracy by always predicting the majority class while learning nothing about the minority. The techniques fall into three groups.

Data-level: random undersampling of the majority class, random oversampling of the minority class, and synthetic oversampling such as SMOTE, which interpolates new minority points between neighbours. Resampling is applied to the training fold only, never to validation or test data, or the metrics become meaningless.

In a real project using Machine Learning, how would you apply, test, or troubleshoot class imbalance techniques?

is inadequate do I add scale_pos_weight or moderate undersampling of the majority (keeping all positives), and I calibrate with isotonic regression afterwards because the claims team consumes the probabilities as risk scores. SMOTE is a last resort here because fraud features are mixed categorical and numeric and interpolated rows are often nonsensical.

Test. The evaluation harness is tested to confirm resampling happens inside imblearn.pipeline.Pipeline so validation folds keep their natural prevalence; a test checks the validation fold’s positive rate is within noise of 0.8%. Model tests assert recall at the fixed alert budget rather than accuracy, and a calibration test checks the Brier score and that the mean predicted probability equals the observed rate.

12

Explain precision recall F1 in the context of Machine Learning.

Precision, recall and F1 are classification metrics computed from the confusion matrix for a chosen positive class and threshold. Precision = TP / (TP + FP): of everything flagged positive, how much was right. Recall = TP / (TP + FN): of all actual positives, how many were caught. F1 = 2 · P · R / (P + R), the harmonic mean, which is high only when both are high and punishes a model that games one at the expense of the other.

The two trade off through the decision threshold: lowering it flags more rows, raising recall and lowering precision. The right balance is a business question. A cancer screen prioritises recall (a missed case is costly, a false alarm leads to a follow-up test); a spam filter prioritises precision (a lost legitimate email is worse than a missed spam). The Fβ score generalises F1 by weighting recall β times as much as precision; F2 favours recall, F0.5 favours precision.

In a real project using Machine Learning, how would you apply, test, or troubleshoot precision recall F1?

Apply. For a content-moderation classifier, the team agreed on two operating points rather than one F1: an automatic-removal threshold where precision must exceed 98%, and a human-review threshold where recall must exceed 90%. Each threshold is picked on the validation set and stored in the model’s metadata, so the serving layer reads the threshold rather than hard-coding 0.5. Multi-class reports use macro F1 as the headline number because the rare “self-harm” class matters more than its support suggests, and the per-class table is reviewed before every release.

Test. Release tests load the frozen thresholds and assert precision at the auto-remove point and recall at the review point on the locked test set, with bootstrap confidence intervals so a two-point drop within noise does not fail the build while a genuine regression does. A slice test computes the same metrics by language and content type, failing if any slice falls more than ten points below the aggregate; aggregate F1 hides minority-language failures.

13

Explain ROC AUC and PR AUC in the context of Machine Learning.

Both summarise a classifier’s ranking quality across all thresholds rather than at one operating point. The ROC curve plots true positive rate (recall) against false positive rate as the threshold sweeps; its area, ROC AUC, equals the probability that a randomly chosen positive is scored higher than a randomly chosen negative. 0.5 is random, 1.0 is perfect ranking. The precision-recall curve plots precision against recall; its area, PR AUC (or average precision), has a random baseline equal to the positive prevalence.

With 1% positives, a model can have ROC AUC 0.95 while precision at useful recall is 10%, because a false positive rate of 2% means twice as many false alarms as true positives. The PR curve exposes that; the ROC curve hides it.

In a real project using Machine Learning, how would you apply, test, or troubleshoot ROC AUC and PR AUC?

Apply. In a lead-scoring model for a sales team, prevalence of conversion is 3% and the team can call only the top 500 leads a week, so PR AUC and precision at k=500 are the model-selection metrics; ROC AUC is logged for comparability with historical models but never decides a release. For the balanced A/B-test classifier of “will this user return in 7 days” (prevalence 48%), ROC AUC is the right headline because both outcomes matter equally. Each experiment logs both metrics plus the curves themselves, so the team can inspect the region of the curve where it actually operates.

Test. A regression test asserts PR AUC on the locked test set does not fall more than 0.02 below the production model’s value, with a bootstrap interval to separate noise from regression. A sanity test asserts ROC AUC on shuffled labels is within 0.52, catching accidental leakage in the evaluation code. Prevalence of the test set is also asserted, because PR AUC comparisons across sets with different prevalence are invalid.

14

Explain confusion matrix in the context of Machine Learning.

A confusion matrix is a table that counts predictions against actual labels, so every error is attributed to a specific pair of classes. For binary classification it has four cells: true positives (TP), false positives (FP), false negatives (FN) and true negatives (TN). Almost every classification metric is a ratio of these cells.

From it: accuracy = (TP + TN) / total, precision = TP / (TP + FP), recall = TP / (TP + FN), specificity = TN / (TN + FP). The matrix depends on the threshold, so changing the cut-off moves counts between FP and FN.

In a real project using Machine Learning, how would you apply, test, or troubleshoot confusion matrix?

Apply. For a support-ticket router with eight categories, I review the row-normalised confusion matrix on every candidate model before looking at any aggregate score. It showed that “billing dispute” and “refund request” were confused 30% of the time; the business agreed to merge them into one queue instead of chasing model accuracy. For a binary loan-default model, I attach costs to the cells (a missed default costs far more than a rejected good applicant) and choose the threshold that minimises expected cost.

Test. The evaluation code is unit-tested on a small hand-made example whose matrix is known, including the row/column orientation, since swapping axes silently exchanges precision and recall. Release gates assert specific cells or derived rates on the locked test set: per-class recall above a floor, and no single off-diagonal cell above a limit. The matrix is also computed per slice (region, channel) and compared with the global one.

15

Explain linear regression assumptions in the context of Machine Learning.

Ordinary least squares gives the best linear unbiased estimates, with valid standard errors and pvalues, under the classical assumptions below. For pure prediction only some matter; for inference all of them do.

Interview takeaway: normality is the least important assumption and exogeneity the most, because only the latter threatens the coefficients themselves.

In a real project using Machine Learning, how would you apply, test, or troubleshoot linear regression assumptions?

Apply. For a pricing-elasticity analysis where the business reads coefficients as “a 1% price rise reduces volume by b%”, all assumptions matter, so I fit a log-log OLS model in statsmodels. For a pure demand-forecast regressor, I treat the assumptions as diagnostics and care mainly about linearity and independence of errors, since predictive accuracy is judged on a hold-out set.

Test. I run a fixed diagnostic bundle after every fit and store the plots as artefacts: - residuals versus fitted values for curvature and funnel shapes; - Breusch-Pagan test for heteroscedasticity; - DurbinWatson statistic (values far from 2 suggest autocorrelation); - variance inflation factors, flagging any above about 5-10; - a Q-Q plot of residuals and Cook’s distance for influential rows.

16

Explain logistic regression in the context of Machine Learning.

Logistic regression is a linear classifier that models the log-odds of the positive class as a linear function of the features: log(p / (1 - p)) = w·x + b. The sigmoid function inverts this to give a probability p = 1 / (1 + e^-(w·x + b)). Despite the name it is a classification model, trained by minimising log loss (cross-entropy), a convex problem with no closed form that is solved iteratively (L-BFGS, SAG, coordinate descent).

It remains a strong baseline for tabular and sparse text data: fast, stable and easy to explain to regulators.

In a real project using Machine Learning, how would you apply, test, or troubleshoot logistic regression?

Apply. In a credit-scoring project where regulators require reason codes, logistic regression is the production model, with WoE-binned or scaled features and L2 regularisation chosen by crossvalidation. Elsewhere it serves as the first baseline for every classification problem: if gradient boosting beats it by only a point of AUC, the simpler model ships. Coefficients feed an adverse-action explanation (“high utilisation raised your odds of default”).

Test. Tests cover predictions and probability quality, not only ranking: AUC and PR AUC on a hold-out, calibration curve and Brier score, and a coefficient-sign test asserting that monotone business rules hold (higher utilisation never lowers risk). Stability is tested by refitting on bootstrap samples and checking that coefficient signs do not flip. The serialised pipeline is round-trip tested so the scaler and model load together.

17

Explain decision trees and random forest in the context of Machine Learning.

A decision tree recursively splits the feature space with threshold rules, choosing at each node the split that most reduces impurity (Gini or entropy for classification, variance for regression). Leaves hold a class distribution or mean. Trees need no scaling, handle mixed feature types and capture interactions naturally, but a deep tree has low bias and very high variance and overfits easily.

A random forest reduces that variance by averaging many decorrelated trees. Each tree is trained on a bootstrap sample of rows (bagging), and at every split only a random subset of features is considered (typically sqrt of the feature count for classification), which decorrelates trees so averaging is effective. Prediction is a majority vote or the mean.

In a real project using Machine Learning, how would you apply, test, or troubleshoot decision trees and random forest?

Apply. For a predictive-maintenance project with 120 mixed numeric and categorical sensor-derived features and a few thousand failures, a random forest is the first strong model: it needs no scaling, tolerates correlated features and gives a usable OOB estimate on day one. A single shallow tree (depth 3-4) is also trained purely as a communication tool, giving maintenance engineers readable rules such as “vibration above X and temperature above Y”.

Test. Tests assert grouped cross-validated PR AUC against a baseline, and that OOB score and crossvalidated score agree within a few points (a large disagreement signals grouped or temporal structure, since the OOB estimate assumes independent rows). A stability test refits with different seeds and asserts the ranking of the top ten permutation importances is mostly preserved. An inference-latency test checks prediction time for a batch against the service limit, because large forests are slow.

18

Explain gradient boosting in the context of Machine Learning.

Gradient boosting builds an additive model sequentially: each new weak learner, usually a shallow decision tree, is fitted to the negative gradient of the loss with respect to the current predictions (the residuals, for squared error), and added with a small step size. F_m(x) = F_(m-1)(x) + η · h_m(x). Where a random forest averages independent deep trees to cut variance, boosting stacks shallow trees to cut bias.

Libraries differ in implementation: XGBoost uses second-order gradients and level-wise growth, LightGBM uses histogram binning with leaf-wise growth for speed, CatBoost uses ordered boosting and native categorical handling. Gradient boosting is typically the strongest approach for structured tabular data, at the price of more tuning and sensitivity to noisy labels than random forests.

In a real project using Machine Learning, how would you apply, test, or troubleshoot gradient boosting?

Apply. For a delivery-ETA model on 50 million rows of tabular features, LightGBM is the production algorithm: fast histogram training, native categorical support and missing-value handling. I train with a low learning rate and early stopping on a time-ordered validation block, use quantile objectives for prediction intervals, and monotone constraints on features like distance, so more distance can never shorten the ETA. Hyperparameters are tuned with Bayesian search over depth, leaves, min_child_samples and regularisation.

Test. I assert that the validation metric improves over a median baseline and over the previous production model on the locked, later-in-time test set. A monotonicity test sweeps distance and checks predictions never decrease. A determinism test fits twice with the same seed and asserts identical predictions, catching accidental nondeterminism from thread counts or unseeded subsampling. Serialisation tests confirm the saved model reloads and scores identically.

19

Explain hyperparameter optimization in the context of Machine Learning.

Hyperparameters are settings fixed before training, such as learning rate, tree depth or regularisation strength, that control model capacity and the learning procedure, as opposed to parameters like weights learned from data. Hyperparameter optimisation (HPO) searches for the configuration that maximises a validation metric, usually estimated with cross-validation.

Practical points: search learning rates and regularisation strengths on a log scale; make the search space reflect plausible ranges; treat the number of trees as an early-stopping outcome rather than a searched value. Tools include scikit-learn’s RandomizedSearchCV and HalvingRandomSearchCV, Optuna and Ray Tune.

In a real project using Machine Learning, how would you apply, test, or troubleshoot hyperparameter optimization?

Apply. For a fraud model retrained monthly, I run Optuna with a TPE sampler and a median pruner for 100 trials, evaluating each configuration with time-ordered folds. The search space is deliberately narrow (learning rate on a log scale, leaves, min_child_samples, subsampling, L2 penalty), tightened from earlier runs. Study results, parameter importance and seeds are logged to the experiment tracker, and the search runs as a scheduled pipeline stage with a fixed compute budget.

Test. Before trusting the search I verify the objective: it must return the same value for the same parameters and seed, and a deliberately bad configuration must score worse than a good one. The winning configuration is re-evaluated on a locked test set and with different CV seeds; if the improvement over defaults is within fold-to-fold noise, I keep the defaults. A budget test asserts the study stops at the trial or time limit.

20

Explain data leakage prevention in the context of Machine Learning.

Data leakage is when information unavailable at prediction time enters the training or evaluation process, producing metrics that the deployed model can never reproduce. Preventing it is a matter of discipline in how data is split, transformed and featurised.

In a real project using Machine Learning, how would you apply, test, or troubleshoot data leakage prevention?

Apply. On a hospital readmission model, I begin with a feature audit: every candidate column gets a documented “available at” timestamp, and anything recorded after the discharge decision is excluded, which removed billing codes and follow-up notes. Patients appear many times, so splits are grouped by patient ID and the test period is later than the training period. Feature joins use as-of logic, so a lab value is only used if its result time precedes the prediction time. All preprocessing sits in one pipeline object.

Test. Automated checks run on every training job: - ID-set intersections between train, validation and test are empty for patients and encounters; - the maximum training timestamp is earlier than the minimum test timestamp; - a single-feature screen trains a tiny model per feature and flags any whose AUC exceeds 0.9 on its own; - a shuffled-label run must score near chance, and a pipeline-fitted-ontrain-only check compares learned statistics with those from the training subset.

21

Explain model explainability SHAP in the context of Machine Learning.

SHAP (SHapley Additive exPlanations) explains an individual prediction by attributing it to features using Shapley values from cooperative game theory. Each feature’s value is its average marginal contribution to the prediction over all subsets of the other features. The attributions sum exactly to the difference between the prediction and the baseline (expected model output): f(x) = base + Σ φ_i.

Limits to state plainly: SHAP explains the model, not the real world, so it shows what the model relies on, not causal effects. Correlated features share credit, which can mislead, and the values depend on the chosen background data and, for tree models, the output space (log-odds rather than probability).

In a real project using Machine Learning, how would you apply, test, or troubleshoot model explainability SHAP?

Apply. In a loan-approval model built with XGBoost, TreeSHAP generates per-decision reason codes: the top three features pushing a score toward decline, translated into customer-readable text by a mapping table. Globally, the beeswarm plot is reviewed with risk and compliance each release to confirm the model relies on sensible drivers and not on proxies for protected attributes. SHAP values are computed offline in batch and stored alongside predictions for audit.

Test. An additivity test asserts base value plus the sum of SHAP values equals the model output (in log-odds) within tolerance for a sample of rows; failure means the explainer and model are out of sync. Sanity tests check direction: raising income in a synthetic row should not produce a more negative income attribution when the model is constrained to be monotone. A stability test compares top-feature rankings across retrains on different seeds, and a leakage test inspects whether the topranked feature is plausible at prediction time.

22

Explain feature selection in the context of Machine Learning.

Feature selection chooses a subset of the available features to keep, to improve generalisation, reduce training and inference cost, and make the model easier to interpret. It differs from dimensionality reduction (PCA), which creates new combined features instead of keeping original ones.

Permutation importance, which measures the score drop when a feature is shuffled on held-out data, is a model-agnostic way to rank features after training, and it avoids the bias of impurity-based importances.

In a real project using Machine Learning, how would you apply, test, or troubleshoot feature selection?

Apply. For a genomics-style problem with 20,000 features and 400 samples, I place feature selection inside the cross-validation pipeline: a variance filter, then L1-regularised logistic regression, with the penalty strength tuned by CV. For a tabular churn model with 600 engineered features, I drop nearconstant and duplicate columns, remove one of each pair correlated above 0.95, and prune the rest using permutation importance on a validation set, keeping the smallest set within one standard error of the best score. This cut inference latency and the number of upstream data dependencies.

Test. I compare the full and reduced models on identical outer folds and assert the reduced model is not worse beyond noise. A selection-stability test repeats selection on bootstrap samples and reports how often each feature is chosen; features appearing in under half of the runs are suspect. A leakage test confirms selection was fitted only on training folds, using a pipeline and an intentionally randomlabel dataset, where a correct pipeline scores near chance.

23

Explain concept drift monitoring in the context of Machine Learning.

Concept drift is a change over time in the relationship between inputs and the target, P(y | x), so a model that was accurate becomes wrong even if the inputs look the same. It is related to, but distinct from, data (covariate) drift, where the input distribution P(x) changes, and label (prior) drift, where P(y) changes. In practice the terms are often used loosely for all three. Drift can be sudden (a policy change), gradual, seasonal or recurring.

A drift alert indicates that something changed, not that the model is broken; the response is investigation, then retraining, recalibration or rollback.

In a real project using Machine Learning, how would you apply, test, or troubleshoot concept drift monitoring?

Apply. For a credit-risk model whose default labels mature after 12 months, performance cannot be measured directly for a year, so I monitor three layers. A daily job computes PSI and null rates for the top 30 features and for the score distribution against the training reference. A weekly job tracks approval rate and the share of applicants in each score band. A monthly job measures AUC and calibration on cohorts whose early-delinquency proxy label has matured. Thresholds route to different actions: a warning goes to the ML channel, a critical alarm pages the owner.

Test. Monitoring code is tested like production code. I replay historical data with synthetic drift (shift a feature mean, flip a category’s frequency, inject nulls) and assert that alerts fire, and that unshifted replays raise none, which measures the false-alarm rate. Historical backtests confirm the detector would have flagged known past events, such as a pandemic-era behaviour change, early enough to be useful.

24

Explain ML deployment and MLOps in the context of Machine Learning.

compared with ordinary software is that behaviour depends on data and code together, so both must be versioned and tested.

time. • Streaming: scoring events as they arrive. • Edge or embedded: model shipped to a device; constrained by size and update cadence.

In a real project using Machine Learning, how would you apply, test, or troubleshoot ML deployment and MLOps?

Apply. For a real-time recommendation service, a scheduled pipeline retrains weekly, evaluates against the current champion on a fixed test set, and registers the model with its data snapshot, code commit and metrics. Promotion to production is a gated step: the model is packaged in a container with pinned dependencies, deployed first in shadow mode, then as a 5% canary, then fully. Features come from a shared feature store so the online and offline paths use identical definitions.

Test. Layers of tests run before release: unit tests on feature code, data-validation checks (schema, ranges, null rates) on training data, model-quality gates versus the champion, and a contract test that the API accepts the documented payload and returns a valid response. Load tests establish p95 latency at expected peak queries per second. During canary I compare score distributions and a business metric between arms, with automatic rollback on error-rate or latency breaches.

25

Explain reproducibility and experiment tracking in the context of Machine Learning.

Reproducibility means that someone can obtain the same model and results from the same inputs, which requires pinning every source of variation: code, data, configuration, environment and randomness. Experiment tracking records these for every training run so results can be compared, audited and re-created.

train/validation/test split. • Configuration: hyperparameters, preprocessing choices, feature list. • Environment: library versions (lock file), Python version, container image, hardware type. • Randomness: seeds for NumPy, Python, framework and data loaders. • Outputs: metrics, curves, plots and the model artefact, linked to the run.

In a real project using Machine Learning, how would you apply, test, or troubleshoot reproducibility and experiment tracking?

Apply. On a team of six building a demand-forecasting product, every training run goes through one entry-point script that reads a versioned config file, seeds all random generators, pulls a named dataset snapshot and logs everything to MLflow. The environment is a Docker image built from a lock file. Models enter the registry only from tracked runs, so for any production prediction the team can find the exact code, data, parameters and metrics that produced it. Notebooks are for exploration; results that matter are rerun through the script.

Test. A CI job reruns a small, fast training configuration twice with the same seed and asserts that metrics and predictions match within a tight tolerance, which catches unseeded randomness and nondeterministic data ordering. A periodic job re-trains a past registered model from its recorded commit, data version and config, and checks the metrics against the logged ones. A tracking test asserts that a run missing the commit hash or data version is rejected at registration.

Answers tailored to your resume and job description

Add your resume and the job description in CrackItX's Settings. When the interviewer asks a Machine Learning question, Claude suggests an answer using your real projects and the role's requirements, in real time.

See pricing →