Short for machine learning operations, MLOps aims to design learning models suitable for deployment in production and then maintain them throughout their lifecycle.
What is MLOps?
MLOps aims to design and maintain machine learning models that can be used in the field. Like DevOps for applications, it involves mastering their entire life cycle. The goal? To take into account deployment constraints from the design and training stages of the model. Following the logic of agile methods, MLOps takes shape through the implementation of learning pipelines combined with model monitoring tools.
The MLOPs engineer is the protagonist. This emerging profession is the product of a cross between the data scientist and the data engineer.
What are the building blocks of MLOps?
MLOps requires the implementation of several bricks aiming to drive the entire machine learning cycle:
- A reusable model store,
- A reusable feature store,
- A continuous integration and delivery (CI/CD) tool,
- A model monitoring and traceability tool,
- A collaborative environment.
What are the tools of MLOps?
Major MLOps tools include:
- DataiKu (proprietary application),
- DataRobot (proprietary application),
- Domino Data (proprietary application),
- Kubeflow (open source application created by Google),
- Metaflow (open source application),
- MLFlow (open source application),
| Techno | Experiment tracking and versioning | AutoML | Orchestration and deployment management | Monitoring | Collaboration |
|---|---|---|---|---|---|
| Dataiku | x | x | x | x | |
| Datarobot | x | x | x | x | |
| Domino Data | x | x | x | x | |
| Kubeflow | x | ||||
| Metaflow | x | ||||
| MLFlow | x | x | |||
| Other solutions often mentioned: Algorithmia (acquired by DataRobot), Cnvrg.io, Polyaxon, Valohai and more recently Comet, Landing AI or Weights & Biases. | |||||
On the cloud provider side, AWS, Google and Microsoft Azure all integrate the MLOps dimension into their respective machine learning platforms, Amazon SageMaker for the first, Vertex for the second and Azure Machine Learning for the third.
Several MLOps training modules are offered online and in science faculties or engineering schools. The MLOps engineer is above all a data scientist. A data scientist training is the key to enter the profession. They must also master the rules of programming and software engineering.
Datascientest is for the moment the only institute to offer a training in MLOps referenced on the Training Application, a training allowing consequently a financing via its training account.
MLOps vs DevOps
DevOps, a contraction of Development (Dev) and Operations (Ops), combines two essential functions: application development and system engineering. The challenge is to take into account deployment constraints from the programming phase and thus improve the quality of the finished product. MLOps is derived from DevOps, but more specifically addresses machine learning oriented applications.