Shikhar Kwatra (IBM), Utpal Mangla (IBM)

ML operationalization (ML Ops) involve a cross matrix collaboration of multiple personas involved in the process of model conceptualization to productization. ML Operationalization overlays paradigm of DevOps on Model Lifecycle management process (CRISP-DM).

According to Forrester, “Creating an ML model is just a starting point. To bring the technology into production service, you need to solve various real-world issues such as building a data pipeline for continuous training, automated validation of the model, version control of the model, creating a scalable serving infrastructure, and ongoing operation of the ML infrastructure with monitoring and alerting.”

Various personas within the MLOps lifecycle may include Data Administrator, Data Steward, Data Engineer, Data Scientist, Business Analyst or alternative roles of users catering to solving a business case using various machine learning models.

“The global MLOps market size is projected to reach USD million by 2027, from USD million in 2020, at a CAGR of % during 2021-2027. With industry-standard accuracy in analysis and high data integrity, the report makes a brilliant attempt to unveil key opportunities available in the global MLOps market to help players in achieving a strong market position.” [1]

In a software development lifecycle, a typical workflow would involve a Development environment, UAT/Pre-prod environment, and Production environment where the final modules post rigorous stress, performance and load testing are deployed for production.

The diagram shown below indicates different environments involved within the MLOps lifecycle. Data acquisition, Data pre-processing and Model development occurs in the Development stage. This stage can involve complex and iterative feature engineering prior to building the model suitable for deployment. Within the UAT/Pre-prod environment, multiple data scientists or model validators may come together to independently validate the model with their blind data. Once the model passes the base performance threshold based on the chosen metrics, the best models are pushed into the Production environment to handle the real traffic. Such optimal models deployed can be invoked using a REST endpoint and consumed by third party applications. The scoring of models in the production environment are critical for measuring the success. There is continuous integration with common repository service for storing model and data artifacts, thereby creating a CI-CD (Continuous Integration-Continuous Deployment) pipeline.

In case the model’s accuracy is reducing over time due to production data drifting from training data (data drift), the model maybe decommissioned. Typically, a couple dozen models are being used in production environment, hence if any model is not performing as per the base threshold set, the real traffic maybe gradually diverted to the other models performing better than the rest. In this way, a continuous pipeline of models being deployed and scored can maintain consistency.

The non-performant models may get trained again on different data sets based on the data drift encountered which led to drifting in the model performance metrics. A consistent process needs to be put in place to ensure the models getting re-trained are versioned properly and stored in the repository to track model lineage in a seamless manner. This set of steps is essential for maximizing the benefits of the models.

Similarly, the data features which are being used consistently for different machine learning or deep learning use cases, may get stored in the common data management repository called “feature stores”. In case a new business case is proposed which may involve reusing some of the existing features, it can easily be fetched from feature store database and used to train the new model accelerating the time in feature engineering and pushing the models into production leading to efficient model operationalization flow. As time progresses, the MLOps market is bound to shine with increasing efforts to streamline the entire end-to-end operationalization process.