Why deploy machine learning models?

Let’s say you’ve created a machine learning model to count the number of cars passing through a particular road using a camera installed on it. The model is initially developed locally by an ML engineer. Once fully developed and tested, it has to move to production otherwise it won’t have any use in real-life. Thus, the need to deploy machine learning models to production.

Before considering your model for deployment, the following elements are necessary:

1. Portability: The concept of "portability" in software development refers to the ability of the software to be easily transferred from one system or machine to another, with low response time and minimal effort required for rewriting.

2. Scalability: "Scalability" refers to the ability of a software model to handle larger workloads without a drop in performance without the need for major redesign or reconfiguration. A scalable model is designed to maintain its performance even when handling large amounts of data or complex tasks.

What is Flask?

Flask is a Python-based open-source web application framework that enables developers to build web applications rapidly and efficiently. It is a lightweight micro-framework that offers essential tools and libraries to create web applications without imposing any strict rules or guidelines.

Flask has a simple and intuitive design that is easy to understand even if you’re new to it. The flexibility and customization options make it an excellent choice for building small to medium-sized applications that require rapid development and deployment.

It supports various HTTP request protocols and has a built-in development server that facilitates the testing and debugging of applications. It also integrates seamlessly with third-party libraries and extensions, providing developers with access to a vast ecosystem of tools and resources that they can leverage to build robust and scalable applications.

Widely used for developing RESTful APIs, microservices, and web applications, Flask’s popularity stems from its simplicity, ease of use, and Pythonic nature. It allows developers to focus on building their applications' core functionalities without worrying about the underlying infrastructure. This makes it an excellent choice for projects with tight deadlines and limited resources.

Microservices and Docker

Before microservices was introduced, software was based on a monolithic architecture. It comprised all the services within a large single application which hampered portability and scalability. Services could not be used independently and hence, deploying such applications was challenging.

Thus, a new concept called microservices was introduced. It can be defined as an approach where an application is composed of several independently working services. They can be invoked independently without affecting the entire architecture and flow of the software. This approach not only resolves scalability issues but has an added advantage: when one service is down due to an error, the entire application doesn’t shut down.

Steps to deploy a machine learning model with Flask and Docker

We will use Docker as a container to deploy the Flask app. A container is similar to a virtual machine except that it does not have its own resources. Rather, it shares them with the host machine. This helps to deploy the app easily by installing all the dependencies.

A self-contained and efficient software package called a Docker container image encompasses all the necessary components to execute an application, including code, runtime, system tools, system libraries, and configurations.

1. Create a machine learning model

In this step, we will create a ML model that will be used for deployment.

# Data Manipulation libraries
import pandas as pd
from sklearn.model_selection  import train_test_split
from sklearn.neural_network import MLPRegressor
import joblib
df = pd.read_csv('house_price.csv')  # Load the dataset
x_train = df[['location', 'bedroom_count', 'locality', 'carpet_area']]
y_train = df[['price']]
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(x_train)
x_train_scaled = scaler.transform(x_train)
x_train_scaled = pd.DataFrame(x_train_scaled, columns=x_train.columns)
X_train, X_test, Y_train, Y_test = train_test_split(df_x_scaled, df_y, test_size = 0.33, random_state = 5)
mlp = MLPRegressor(hidden_layer_sizes=(60), max_iter=1000)
mlp.fit(X_train, Y_train)
Y_predict = mlp.predict(X_test)
#Saving the machine learning model to a file
joblib.dump(mlp, "model/price_model.pkl")

We load the dataset named “house_price.csv” and perform basic preprocessing steps like scaling and splitting the dataset into training and testing sets. We then pass the training set into the MLP regressor for training. Finally, we save the trained model using the joblib library in pickle format.

2. Create a REST API with Flask

Now, we will create a Flask API that will be called later for inferencing.

#importing necessary libraries
from flask import Flask, jsonify, request
import pandas as pd
import joblib

app = Flask(name)
@app.route("/predict", methods=['POST'])
def do_prediction():
json = request.get_json()
#loading saved model here in this python file
model = joblib.load('model/rf_model.pkl')
#creating data frame of JSON data
df = pd.DataFrame(json, index=[0])
from sklearn.preprocessing import StandardScaler
#performing preprocessing steps
scaler = StandardScaler()
scaler.fit(df)

x_scaled = scaler.transform(df)

x_scaled = pd.DataFrame(x_scaled, columns=df.columns)
y_predict = model.predict(x_scaled)

res= {"Predicted Price of House" : y_predict[0]}
return jsonify(res)

if name == "main":
app.run(host='0.0.0.0')

Next, we will create a requirements.txt file such that all the dependencies can be installed in a single command. Below is the code for the same:

pip freeze> requirements.txt

Once you have created the Flask API, the code above has to be pushed to the GitHub repository so that it can be cloned when you dockerize the entire code.

3. Create a Docker Image

After creating the service, the initial step will involve defining the Docker image, which is critical for the process. Defining the Dockerfile contents is essential to include all the necessary dependencies and copy the application's content into the container.

Below is the content to be included in the Docker file:

#Using the base image with Python 3.10
FROM python:3.10

#Set our working directory as app
WORKDIR /app
#Installing Python packages through requirements.txt file
RUN pip install -r requirements.txt
Copy the model's directory and server.py files
ADD ./models ./models
ADD server.py server.py
#Exposing port 5000 from the container
EXPOSE 5000
#Starting the Python application
CMD ["gunicorn", "--bind", "0.0.0.0:5000", "server:app"]

4. Run Docker

Here, we will build a Docker container for the service. Note that it is necessary to install Docker before this step is executed.

#Cloning the repo from Github in your local
git clone https://github.com/harsha89/ml-model-tutorial.git
#Building the docker image
docker build -t ml-model

Once you achieve this, you can push this code to the Docker repository which can be used to run the application from anywhere.

Conclusion

We’ve discussed why ML models need to be deployed to production and how to do so using Docker and Flask. Without deployment, trained models cannot be used for inference for real-time data. To deploy any service to production, two key factors are important, i.e., scalability and portability. Microservices architecture helps introduce these two factors and allows each service to run in a container, such as Docker, as an independent service.