Survey of Machine Learning with Hugging Face

This module gives you a practitioner vocabulary for what supervised learning is optimizing, how to tell underfitting from overfitting, and how to choose and align evaluation metrics with the host score. Key concepts: the supervised learning story as function approximation, bias and variance as the language of model diagnosis, competition metrics (AUC, log loss, RMSE, and related choices), and the iterate-with-care loop from EDA through validation. By the end of this module, you will have: a one-page written narrative of your own baseline setup that names the task, metric, and what train versus validation behavior would mean for the next change.

This module treats the Hugging Face Hub and libraries as the shared plumbing for loading data, models, and metrics in modern ML workflows, including the path from a Kaggle notebook to versioned assets. Key concepts: Hub model and dataset pages, the Datasets and Evaluate libraries for IO and metrics, scikit-learn interoperability via skops, and how AutoTrain and Spaces relate to quick baselines. By the end of this module, you will have: a small notebook that loads a public dataset through Datasets, runs one metric with Evaluate, and records the exact model or dataset id you would cite in a write-up.

This module focuses on exploratory analysis as the main lever for competition rank: most lift comes from understanding the table before you trust any score. Key concepts: the five-question EDA pass over shape and leakage risk, adversarial train-versus-test checks for shift, group and ID structure that breaks naive cross-validation, and simple visuals that make distribution mismatch obvious. By the end of this module, you will have: a short EDA report for one competition-style table with at least one adversarial validation AUC and a written note on whether stratified, grouped, or time-based splits are implied.

This module builds the tabular feature factory mindset: you supply compositional structure that tree models and linear heads can exploit, instead of hoping the algorithm invents it from raw columns alone. Key concepts: groupby aggregations and transforms on entity keys, out-of-fold target and impact encodings, numeric transforms and scaling choices aligned with the metric, and keeping lineage so validation stays honest. By the end of this module, you will have: a feature notebook that adds at least one groupby-based feature set and uses OOF or fold-scoped encoding for any target-derived column, with a table listing raw columns each engineered feature depends on.

This module ties classical tabular power to the gradient-boosted decision tree family and its modern extensions, from histogram optimization to categorical handling and foundation-style tabular models. Key concepts: how boosting composes weak trees under a loss, the practical XGBoost, LightGBM, and CatBoost tradeoffs, early stopping and regularization knobs, and when TabPFN-style in-context models complement long GBDT runs. By the end of this module, you will have: OOF prediction files (or a documented pilot) for at least two GBDT variants plus a written comparison of when you would add TabPFN to the same fold protocol.

This module makes honest validation the default: your cross-validation design is the lens through which every feature and model claim must pass. Key concepts: K-fold, stratified, grouped, and time-based splits, out-of-fold predictions for unbiased training-row scores and for stacking, target leakage and test contamination in preprocessing, and when public leaderboard movement should trigger a protocol audit instead of more tuning. By the end of this module, you will have: a working OOF array for a baseline model on a full training set with a split that matches the problem (IID, group, or time), plus a one-page leakage checklist you applied to your feature list.

This module connects hyperparameters to the bias-variance path you already know, then treats search as a disciplined experiment: bounded budget, clear metric, and logs you can compare weeks later. Key concepts: what major GBDT parameters change in the fit, grid versus random versus Bayesian (TPE) search with Optuna, when extra tuning is wasted because validation is wrong or features starve the model, and experiment tracking with Weights and Biases or equivalent. By the end of this module, you will have: a small Optuna study on one booster with stored best params, or a written rationale for why you stopped at a hand-tuned grid, plus a run log that includes seed, data fingerprint, and score.

This module explores agentic workflows in which an LLM accelerates code and analysis while you remain responsible for validation, leakage, and the competition contract. Key concepts: human-in-the-loop review of generated CV and data paths, a four-step loop from EDA through model combination that mirrors Grandmaster practice, risks of plausible but wrong code, and tools such as smolagents for wiring agents. By the end of this module, you will have: a short prompt or agent script for one real step (for example EDA or baseline) plus a written list of which outputs you would never trust without running on your own split.

This module applies the Transformers and token-classification stack to text competitions, including efficient fine-tuning and embedding-based hybrids with tabular models. Key concepts: tokenizer and model loading patterns, full versus PEFT (LoRA) fine-tuning under VRAM limits, DeBERTa and similar encoders for classification, masked-language-model adaptation when rules allow, and Sentence Transformers for dense text features. By the end of this module, you will have: one fine-tuning or feature pipeline script that exports OOF and test predictions with frozen fold indices and a note on which heads are used for the task.

This module covers image modeling with the timm catalog and Hugging Face vision checkpoints, with augmentation and test-time augmentation as first-class parts of the model. Key concepts: transfer learning from ImageNet-scale pretraining, Albumentations pipelines and order of operations, choosing backbones versus ensembling diverse pretrained families, and TTA as cheap inference-time averaging. By the end of this module, you will have: a training script or notebook outline that freezes or fine-tunes one timm model, applies a documented augment policy, and saves OOF or validation scores you can compare across two backbones.

This module treats ordered data as a different validation problem: random K-fold is often wrong, and features must respect the arrow of time. Key concepts: purged or gap splits, TimeSeriesSplit-style cross-validation, lag and rolling statistics with clear causality, and boosting on hand-built temporal features versus classic univariate baselines. By the end of this module, you will have: a time-aware CV object applied to at least one series or panel table, with one rolling feature whose definition states which past rows it is allowed to see.

This module turns diverse base models into calibrated combinations: averaging, learned weights, and stacked second-level learners trained only on out-of-fold predictions. Key concepts: correlation versus diversity of errors, blending versus stacking, linear and nonlinear metas on OOF matrices, hill climbing and greedy selection, and why multi-level stacks need nested OOF discipline. By the end of this module, you will have: a stack or blend that beats your best single-model OOF on a saved OOF design matrix, with row alignment checks documented for every base column.

This module covers late-competition moves that only make sense when your validation story is already sound: self-training, distillation, full-data retraining, and sharper shift detection. Key concepts: pseudo-labels and confidence filters, teacher-student and knowledge distillation patterns, retraining on all permitted training rows with seed and model diversity, and adversarial validation for train-test shift and feature audits. By the end of this module, you will have: a short protocol write-up for one technique (for example one pseudo-label round with threshold and cap) that states what would abort the run if OOF regresses.

This module pulls process and career threads together: how to run a competition like a product sprint, how to read leaderboards and forums without overfitting the public split, and how to make work visible in a portfolio. Key concepts: change-one-thing discipline and run ledgers, forum and write-up triage, submission budget and when to stop, and publishing models or Spaces as evidence of skill. By the end of this module, you will have: a capstone or portfolio artifact that links one Hub asset or competition submission to a short narrative a hiring manager or teammate could follow, plus your personal list of what you would do differently on the next event.

A comprehensive reference of the 50+ key terms, acronyms, and concepts used throughout this course. Use this module as a lookup when you encounter unfamiliar terminology in any lesson or lab. Terms are organized alphabetically with cross-references to the module where each concept is introduced.