MLflow is an open-source tool that records every machine learning training run (its parameters, metrics, code version and output files), lets you compare runs side by side, packages trained models in a standard format, and manages model versions through a registry. Combined with DVC or similar tools for versioning data, it makes ML work reproducible: you can always answer "which data, code and settings produced this model?".

Why do ML projects need experiment tracking?

A typical project involves dozens or hundreds of training runs: different features, algorithms, hyperparameters and data versions. Without tracking, results end up scattered across notebook cells, file names like model_final_v3_really_final.pkl and memory. Weeks later nobody can reproduce the best model, explain why it was chosen, or tell whether a new model is genuinely better. Experiment tracking is the lab notebook of machine learning, and in a team it's what lets colleagues build on each other's work instead of repeating it.

What are MLflow's main components?

ComponentWhat it does
TrackingLogs parameters, metrics, tags and artifacts for each run; web UI to compare runs
ModelsSaves models in a standard format with their environment, loadable by many serving tools
Model RegistryVersions models, records lineage, and marks which version is approved for use
ProjectsPackages code with its environment so runs can be reproduced
LLM evaluation and tracingNewer features to log prompts, outputs and evaluation results for LLM applications

What should you log for every run?

  • Parameters: model type, hyperparameters, feature set, random seed.
  • Metrics: validation scores (and per-epoch curves), training time, inference latency.
  • Data version: a dataset hash or DVC version, plus row counts.
  • Code version: the Git commit (MLflow can record it automatically).
  • Artifacts: the model file, plots such as confusion matrices, feature importance, sample predictions.
  • Environment: library versions, so the model can be loaded later.

In code, this is a few lines: start a run with mlflow.start_run(), call mlflow.log_param(), mlflow.log_metric() and mlflow.log_artifact(), and save the model with a flavour such as mlflow.sklearn.log_model(). Autologging for popular libraries records many of these automatically.

How do you compare runs?

The MLflow UI lists runs in a table you can sort and filter by any parameter or metric, and plots metrics across runs. Typical questions it answers in seconds: which learning rate gave the best validation F1? Did adding the new features help on average across seeds? Which run is the current best, and what data did it use? Tagging runs (for example "baseline", "feature-set-v2", "candidate") keeps large experiment lists navigable.

Example comparison view: validation F1 for five tracked runs
Baseline (logistic regression)0.61
Random forest0.66
XGBoost, default0.68
XGBoost, tuned0.71
XGBoost + new features0.74

Illustrative numbers showing how a tracking UI makes progress visible; not results from a real project.

What is the model registry for?

Once a run produces a good model, you register it under a name, such as "churn-model". Each registration becomes a new version with a link back to the run that produced it. Teams then mark which version is the current candidate or the one in production (recent MLflow versions use aliases such as "champion" and "challenger"). Serving code loads "the production version of churn-model" rather than a file path, so promoting or rolling back a model is a registry change rather than a code change, and there's always a record of who changed what.

How does data versioning fit in?

Code lives in Git, but datasets are often too large for it. DVC (Data Version Control) stores small pointer files in Git and the actual data in cloud or local storage, so each Git commit identifies exactly which data version it used. DVC can also define pipelines (prepare data → train → evaluate) and rerun only the steps whose inputs changed. Logging the DVC data version as an MLflow parameter connects every model to its exact data.

How does this connect to deployment and monitoring?

An MLflow model can be served directly with built-in tools, loaded into your own API (see deploying models with FastAPI), or packaged into a container. A CI/CD pipeline can train, evaluate against a threshold, register the model and deploy it automatically if it beats the current production version. After deployment, monitoring compares live input data and predictions with the training data to detect drift, which triggers retraining, a new tracked run, and the cycle repeats.

What good habits make tracking useful?

  • Log from the first experiment, not only once things look promising.
  • Fix and log seeds and run important comparisons with several seeds.
  • Name experiments by question ("does adding location help?"), not by date.
  • Never overwrite artifacts; create new runs.
  • Record negative results: knowing what didn't work saves teammates time.
  • Keep a final, untouched test set and log its result only for chosen models.

How do teams set up MLflow?

For learning and small projects, MLflow runs locally and stores runs in a folder or a local database; you start the UI with one command. Teams run a shared tracking server backed by a database (such as PostgreSQL) for run metadata and object storage (such as an S3-compatible bucket) for artifacts, with authentication in front. Managed offerings on major cloud platforms remove the server maintenance. Whatever the setup, agree on naming conventions for experiments and registered models early, because a shared tracking server quickly fills up with hundreds of runs.

Does MLflow help with LLM applications?

Yes. LLM projects have their own "experiments": different prompts, retrieval settings, models and temperatures. Logging each configuration with its evaluation scores on a fixed test set, plus sample outputs, brings the same discipline to prompt and RAG development. Newer MLflow versions add tracing of LLM calls, which shows the prompt, retrieved context, tool calls and response for each request, making it much easier to debug why an answer went wrong.

What does a tracked project look like in practice?

Imagine a team predicting which learners need extra support. Each experiment is one question: "does adding attendance streak features help?" Inside it, every run logs its parameters, the data version from DVC, the Git commit and validation metrics across five folds. After a week, the team opens the comparison view, sorts by mean F1, checks that the best run's advantage is larger than the fold-to-fold spread, and registers it as a new version of the model. The serving API loads whichever version holds the "champion" alias. When a later run beats it, the team moves the alias, and the change is recorded with a note explaining why. Months later, an auditor's question about how the model was built can be answered in minutes.

What are alternatives to MLflow?

Weights & Biases and Comet are popular hosted tools with polished dashboards; Neptune and ClearML offer similar tracking; cloud platforms provide their own experiment and model management. The concepts transfer directly: runs, parameters, metrics, artifacts, registries and lineage. Learning MLflow first is a good choice because it's open source, widely used, and runs locally with no account.

How do you learn MLOps tools?

Take a project you've already built, add MLflow logging to every run, version its data with DVC, register the best model and load it from the registry in a small API. MLflow, DVC, model serving with FastAPI and Triton, CI/CD and monitoring make up Program Zero's MLOps and cloud deployment course; read MLOps explained for the overall picture.