MLOps Simplified with MLflow: A Comprehensive Overview
Written on
MLflow is a widely-used open-source framework that is vital for the successful implementation of MLOps, which stands for Machine Learning Operations. At its core, MLOps applies DevOps principles to the machine learning lifecycle, focusing on the optimization of the development, deployment, and management of machine learning models.
The machine learning community is currently striving to create a standardized model for developing machine learning projects. As a result, numerous projects within machine learning and data science face organizational challenges, making it difficult to reproduce outcomes. Traditionally, these projects have been executed in an unstructured manner. To aid machine learning practitioners throughout the development lifecycle, a new framework has been proposed: the Cross-Industry Standard Process for Machine Learning applications with Quality assurance methodology (CRISP-ML(Q)). This article explores the main stages of the machine learning development process model. While the stages follow a particular order, it's important to recognize that machine learning workflows are typically iterative and exploratory, often requiring revisiting earlier steps based on results from later phases.
The CRISP-ML(Q) process model consists of six key phases:
- Business Understanding and Data Understanding: This initial phase involves grasping the business challenge and the data at hand. It includes defining business goals, identifying success metrics, and examining the data to comprehend its properties and machine learning potential.
- Data Preparation: This stage mirrors traditional lifecycles, concentrating on data cleansing and preparation for modeling. It involves addressing missing values, resolving inconsistencies, and possibly transforming the data into an appropriate format.
- Modeling and Tuning: In this phase, various machine learning algorithms are selected and tested. Models are trained using the prepared data, and techniques like hyperparameter tuning are employed to maximize performance.
- Evaluation: This critical phase assesses the effectiveness of the trained models. Various evaluation metrics are utilized to evaluate the model's performance on unseen data and to compare different models for optimal selection.
- Deployment: Once a model demonstrates satisfactory performance, it can be deployed into a production environment. This entails packaging the model, integrating it with existing systems, and establishing infrastructure for prediction services.
- Monitoring and Maintenance: Continuous monitoring of model performance is crucial, similar to traditional lifecycles. This involves tracking accuracy, addressing any performance degradation, and retraining the model with new data as necessary to ensure ongoing effectiveness.
CRISP-ML offers several benefits:
- Standardization: It establishes a common language and framework for machine learning projects, promoting collaboration and knowledge sharing among teams.
- Structured Approach: Adhering to its phases ensures a thorough and organized development process, minimizing the risk of overlooking essential steps.
- Enhanced Communication: The terminology used in CRISP-ML aids in effective communication between data scientists, business stakeholders, and other project contributors.
While CRISP-ML is not a strict formula, it serves as a valuable guideline for crafting robust and effective machine learning models.
For each stage in the process model, CRISP-ML(Q) emphasizes a quality assurance framework that encompasses several essential elements. These include defining requirements and constraints, such as performance benchmarks and data quality standards, executing specific tasks like selecting machine learning algorithms and conducting model training, identifying potential risks that might impede the success of the machine learning application, and implementing quality assurance strategies to address these risks when necessary. Techniques like cross-validation and thorough documentation of both processes and results are examples of such strategies.
Key Components of MLflow
MLflow consists of several essential components that collectively facilitate the machine learning lifecycle. These elements work together to enhance the development, experimentation, deployment, and monitoring of machine learning models. The core components of MLflow include:
- MLflow Tracking:
- Function: Keeps track of experiments throughout the machine learning lifecycle, serving as the central repository for recording details about training runs.
- Key Features:
- Logs parameters used during training (e.g., learning rate, number of epochs).
- Captures metrics to assess model performance (e.g., accuracy, loss).
- Tracks artifacts produced during runs (e.g., trained model files, code versions).
- Offers an API and UI for interaction with this logged data.
- MLflow Projects:
- Function: Guarantees the reproducibility of machine learning projects.
- Key Features:
- Packages code, environment, and dependencies into a reusable structure.
- Allows for consistent results when running the same project across different machines or environments.
- Encourages collaboration by simplifying project sharing among team members.
- MLflow Models:
- Function: Streamlines the management of trained machine learning models.
- Key Features:
- Saves models in a platform-agnostic format for versatility.
- Facilitates version control to track changes and allow rollbacks.
- Permits deployment across various serving environments.
- Simplifies the process of serving predictions from trained models.
- MLflow Model Registry (Optional):
- Function: Offers a centralized repository for enhanced model governance (not essential for basic MLflow usage).
- Key Features:
- Stores and manages different versions of trained models.
- Provides stage transitions for models (e.g., development, staging, production).
- Implements model approval workflows for controlled deployment.
- Enhances governance and accountability in production settings.
These components can function independently for specific tasks, but their true effectiveness is realized when used in combination. By integrating MLflow Tracking, Projects, and Models, you can establish a robust and reproducible machine learning workflow that fosters collaboration and success in your ML initiatives.
Together, these components create a comprehensive platform for overseeing the entire machine learning lifecycle, from experimentation and development through to deployment and monitoring. With MLflow, data scientists and machine learning engineers can streamline their workflows, boost productivity, and enhance collaboration and reproducibility across their organizations.
Setting Up MLflow
Establishing MLflow involves several steps, including installation, configuration, and potential integration with other tools or platforms. Here’s a general guide for setting up MLflow:
Install MLflow
You can easily install MLflow using Python’s package manager, pip. Execute the following commands in your terminal or command prompt:
pip install virtualenv
python -m venv venv
source venv/bin/activate
pip install mlflow
Start MLflow Tracking Server (if applicable)
If you installed the MLflow tracking server, you can initiate it by executing the command below in your terminal:
mlflow ui
This command launches the MLflow tracking server locally, allowing access to the web UI by navigating to http://localhost:5000 in your web browser.
Clone the MLflow Repository
Download the MLflow repository using:
git clone https://github.com/mlflow/mlflow
Navigate into the examples directory within your cloned MLflow repository, as this will serve as your working directory for the tutorial. Running directly from your MLflow clone may lead to utilizing the source version instead of your PyPI installation.
Running the MLFlow Experiment
Before initiating your experiment, ensure that your tracking URI is set to your MLFlow server to log your results effectively. You can set the environment variable in your terminal:
export MLFLOW_TRACKING_URI=http://localhost:5000
To define the URI for a single command, use:
MLFLOW_TRACKING_URI=http://localhost:5000 mlflow run --experiment-name=experiment_name_here
Once your server and URI are configured, you’re ready to run the experiment! In the directory containing your example, execute the command:
mlflow run . --experiment-name=sklearn_lr_example
To assign a name to this specific run, append:
--run-name name_your_run_here
cd mlflow/examples/sklearn_logistic_regression/
export MLFLOW_TRACKING_URI=http://localhost:5000
# if error Can't Find pyenv
curl https://pyenv.run | bash
python -m pip install virtualenv
PATH="$HOME/.pyenv/bin:$PATH"
mlflow run . --experiment-name=lr_example --run-name=first_run
MLFlow Dashboard and Model Serving
After completing your sample run, you can review how to access this experiment through the MLFlow server dashboard.
Serving the Model
To serve this model, copy the “Run ID” or the “Full Path” of the model artifact for the specific run you wish to host.
Now, serve the model by executing one of the commands below. This command will host the model on your machine using port 1234. It's crucial to specify the port number, as the default port is 5000, which is used by the MLFlow server. Feel free to adjust this port number as necessary:
If you used the “Run ID”:
mlflow models serve -m runs:/paste_your_ID_here --port 1234
If you used the “Full Path”:
mlflow models serve -m mlflow-artifacts:/.../.../artifacts/model --port 1234
You’re now ready to test your model by sending a request. For a quick test, you can use curl to send a DataFrame to the model:
curl -d '{"dataframe_split": {"columns": ["x"], "data": [[5],[-1],[0],[1],[2],[1]]}}' -H 'Content-Type: application/json' -X POST localhost:1234/invocations
Model Registration and Version Control
An additional important feature of MLflow is the model registry. Registering models in MLflow is a critical step in managing models, allowing for:
- Versioning: Each registered model is assigned a unique version number, enabling tracking of changes over time and allowing rollbacks to earlier versions when necessary.
- Tracking: Registered models can be monitored, allowing for logging and visualization of performance, inputs, and outputs in the MLflow UI, enhancing understanding of the model's effectiveness and usage.
- Deployment: Registered models can be effortlessly deployed into production. MLflow provides a straightforward API for deployment, facilitating integration into existing systems.
- Reproducibility: Models can be associated with metadata such as descriptions, authors, and run information, simplifying the reproduction of experiments and the understanding of the model's context.
- Collaboration: Registered models can be shared, enabling team members to use and enhance the model easily.
To register a model via your server dashboard, revisit one of your runs and click the "Register Model" button to create a new model or associate this run with an existing model.
You will see this in the "Models" section of your server dashboard, showing an overview of the latest version, variants in staging, and the current production model. To register a production model, select the model name.
Click on the version you wish to designate as your production model.
You can now use the same MLflow command previously employed to serve the model, referencing it by its name and version:
mlflow models serve -m "models:/example_model/1" --port 1234
For further information, consult MLflow Projects.
If you found this article helpful, please consider supporting the author by clapping below! Thank you for reading!
Oleh Dubetcky | [LinkedIn](https://www.linkedin.com/in/oleh-dubetcky/)