Digits Recognizer using Python and React.
Build backend with Flask.
Build backend with Flask.
Do you have any questions? Contact us!
I agree the Terms of Service
published April 4, 2018
It is the second part of my long journey to creating the web application for handwritten digits recognition. Here I'll explain how to integrate our trained model into the Flask application.
It is the second part of my long journey to creating the web application for handwritten digits recognition. Here I'll explain how to integrate our trained model into the Flask application.
#Environment setup
As we build the application from scratch, we need to create a virtual environment to encapsulate all our dependencies. Let's create python3 virtual environment.
python3 -m venv venv source venv/bin/activate
As we build the application from scratch, we need to create a virtual environment to encapsulate all our dependencies. Let's create python3 virtual environment.
python3 -m venv venv source venv/bin/activate
Now we have the environment and we can install all needed dependencies.
pip install flask flask-script scipy scikit-image numpy pillow
That's all about setting up the environment.
#Configure application
We need an option to run the application without setting the env variables every single time. We can achieve this goal by using the flask-script package. At first, we should create a file called manage.py
touch manage.py
The next step is getting this manage.py ready.
pip install flask flask-script scipy scikit-image numpy pillow
That's all about setting up the environment.
#Configure application
We need an option to run the application without setting the env variables every single time. We can achieve this goal by using the flask-script package. At first, we should create a file called manage.py
touch manage.py
The next step is getting this manage.py ready.
from flask_script import Manager
from app import create_app
app = create_app()
manager = Manager(app)
@manager.command
def runserver():
app.run(debug=True, host='0.0.0.0', port=5000)
if __name__ == '__main__':
manager.run() As you see, there is an unknown function create_app inside the app package. Now we'll make this function known. I start by creating the package.
mkdir app touch app/__init__.py
And then make the target function
mkdir app touch app/__init__.py
And then make the target function
from flask import Flask
def create_app():
app = Flask(__name__)
return app Now our app is ready and can be run by command.
python3 manage.py runserver
#Add storage
As we deal with the kNN classifier, we need to remember the clusters after the training process. This simplifies the way of storing the serialized version of the classifier in a file. Let's create this file.
mkdir storage touch storage/classifier.txt
#Create the settings file
Finally, we need to create a settings file where we'll have the application's base directory paths and the classifier storage.
touch settings.py
The file's content is
python3 manage.py runserver
#Add storage
As we deal with the kNN classifier, we need to remember the clusters after the training process. This simplifies the way of storing the serialized version of the classifier in a file. Let's create this file.
mkdir storage touch storage/classifier.txt
#Create the settings file
Finally, we need to create a settings file where we'll have the application's base directory paths and the classifier storage.
touch settings.py
The file's content is
import os
BASE_DIR = os.getcwd()
CLASSIFIER_STORAGE = os.path.join(BASE_DIR, 'storage/classifier.txt')#Make handlers
The first step is to specify processors for incoming requests. We'll develop a SPA so the first processor will always return an index.html page.
The first step is to specify processors for incoming requests. We'll develop a SPA so the first processor will always return an index.html page.
def create_root_view(app):
@app.route('/', defaults={'path': ''})
@app.route('/<path:path>')
def root(path):
return render_template("index.html")Besides this view, we also must create an API endpoint for predictions. For now, we'll just pass handling the request but return here afterward.
class PredictDigitView(MethodView):
def post(self):
pass#Construct the prediction logic
For our prediction view, we need two things: an image processor and a kNN classifier. As we've already trained our classifier, we just have to realize the image processing.
The image processor will be represented by lots of functions to convert our image from data URI to the flat numpy array of the 8x8 grayscale image with intensity from 0 to 16(like the original ones from the digits dataset).
First, we need to convert the image from the data URI to the Image object.
For our prediction view, we need two things: an image processor and a kNN classifier. As we've already trained our classifier, we just have to realize the image processing.
The image processor will be represented by lots of functions to convert our image from data URI to the flat numpy array of the 8x8 grayscale image with intensity from 0 to 16(like the original ones from the digits dataset).
First, we need to convert the image from the data URI to the Image object.
import numpy as np
from skimage import exposure
import base64
from PIL import Image
from io import BytesIO
def data_uri_to_image(uri):
encoded_data = uri.split(',')[1]
image = base64.b64decode(encoded_data)
return Image.open(BytesIO(image))Then because of the specificity of our front end(will see it later) we need to replace the image background from transparent to white.
def replace_transparent_background(image):
image_arr = np.array(image)
alpha1 = 0
r2, g2, b2, alpha2 = 255, 255, 255, 255
red, green, blue, alpha = (
image_arr[:, :, 0],
image_arr[:, :, 1],
image_arr[:, :, 2],
image_arr[:, :, 3]
)
mask = (alpha == alpha1)
image_arr[:, :, :4][mask] = [r2, g2, b2, alpha2]
return Image.fromarray(image_arr) Another thing to do is avoid the white frame of the image and replace it with the small one just like in the test data. We'll do it in 2 steps:
- Remove the whole frame
def crop_image_frame(image, color=255):
image_arr = np.array(image)
cropped_image_arr = image_arr[~np.all(image_arr == color, axis=1)]
cropped_image_arr = cropped_image_arr[:, ~np.all(cropped_image_arr == color, axis=0)]
return Image.fromarray(cropped_image_arr)- Add the frame that we need
def pad_image(image, height=0, width=30, color=255):
return Image.fromarray(np.pad(
np.array(image),
((height, height), (width, width)),
'constant',
constant_values=color
)) After that let's resize the image to 8x8 format.
def resize_image(image):
return image.resize((8, 8), Image.ANTIALIAS) The original images' features show the intensity of gray color from 0 to 16. To reach this gain, we have only the rescale_intensity function, which just changes our image scale. If we have a pixel with the value 255(white), it will be scaled into 16. But the intensity of the gray color is 0 in white. That's why we need to invert white to 0(black) and only after that change the intensity scale.
def white_to_black(image):
image_arr = np.array(image)
image_arr[image_arr > 230] = 0
return Image.fromarray(image_arr)
def to_flat_grayscaled_image_arr(image):
image_arr = np.array(image)
image_arr = exposure.rescale_intensity(image_arr, out_range=(0, 16))
return image_arr.flatten()Now we can combine all the image utils together. The only addition is to convert the image from RGB to L format. So all RGB pixels will be aggregated by the formula: L = R _ 299/1000 + G _ 587/1000 + B * 114/1000.
def to_classifier_input_format(data_uri):
raw_image = data_uri_to_image(data_uri)
image_with_background = replace_transparent_background(raw_image)
grayscaled_image = image_with_background.convert('L')
cropped_image = crop_image_frame(grayscaled_image)
padded_image = pad_image(cropped_image)
inverted_image = white_to_black(padded_image)
resized_image = resize_image(inverted_image)
return np.array([
to_flat_grayscaled_image_arr(resized_image)
])#Handle the request
The first thing to do is to create the repo for getting and updating our classifier.
The first thing to do is to create the repo for getting and updating our classifier.
import pickle
class ClassifierRepo:
def __init__(self, storage):
self.storage = storage
def get(self):
with open(self.storage, 'rb') as out:
try:
classifier_str = out.read()
if classifier_str != '':
return pickle.loads(classifier_str)
else:
return None
except Exception:
return None
def update(self, classifier):
with open(self.storage, 'wb') as in_:
pickle.dump(classifier, in_) Also, we need a factory to create the fitted classifier.
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier
class ClassifierFactory:
@staticmethod
def create_with_fit(data, target):
model = KNeighborsClassifier(n_neighbors=3)
model.fit(data, target)
return model It's almost done. The only thing to finish our view is to gather all the things above into a service.
from sklearn.datasets import load_digits
from app.classifier import ClassifierFactory
from app.utils import to_classifier_input_format
class PredictDigitService:
def __init__(self, repo):
self.repo = repo
def handle(self, image_data_uri):
classifier = self.repo.get()
if classifier is None:
digits = load_digits()
classifier = ClassifierFactory.create(
digits.data, digits.target
)
self.repo.update(classifier)
x = to_classifier_input_format(image_data_uri)
prediction = classifier.predict(x)[0]
return predictionSo, let's add this service in our view.
class PredictDigitView(MethodView):
def post(self):
repo = ClassifierRepo(CLASSIFIER_STORAGE)
service = PredictDigitService(repo)
image_data_uri = request.json['image']
prediction = service.handle(image_data_uri)
return Response(str(prediction).encode(), status=200) And initialize handlers by calling this function inside the create_app.
def init_urls(app):
app.add_url_rule(
'/api/predict',
view_func=PredictDigitView.as_view('predict_digit'),
methods=['POST']
)
create_root_view(app) So the final version of __init__.py inside the app folder
from flask import Flask
from .urls import init_urls
def create_app():
app = Flask(__name__)
init_urls(app)
return app#Testing
Now you can test our app. Let's run the application.
python3 manage.py runserver
And send an image in base64 format. You can do it by downloading this image, converting it to base64 using this resource, copying the code, and saving it in the file called test_request.json. Now we can send this file to get a prediction.
curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n'
You should see the following output.
\(venv) Teimurs-MacBook-Pro:digits-recognizer teimurgasanov$ curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n' HTTP/1.1 100 Continue HTTP/1.0 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 1 Server: Werkzeug/0.14.1 Python/3.6.3 Date: Tue, 27 Mar 2018 07:02:08 GMT 4
As you see, our web app correctly detected that it is 4.
#Final result
You can find the code from this article in my Github repository.
Now you can test our app. Let's run the application.
python3 manage.py runserver
And send an image in base64 format. You can do it by downloading this image, converting it to base64 using this resource, copying the code, and saving it in the file called test_request.json. Now we can send this file to get a prediction.
curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n'
You should see the following output.
\(venv) Teimurs-MacBook-Pro:digits-recognizer teimurgasanov$ curl 'http://localhost:5000/api/predict' -X "POST" -H "Content-Type: application/json" -d @test_request.json -i && echo -e '\n\n' HTTP/1.1 100 Continue HTTP/1.0 200 OK Content-Type: text/html; charset=utf-8 Content-Length: 1 Server: Werkzeug/0.14.1 Python/3.6.3 Date: Tue, 27 Mar 2018 07:02:08 GMT 4
As you see, our web app correctly detected that it is 4.
#Final result
You can find the code from this article in my Github repository.
Did you like this article?
Share article on social networks
Originally published at teimurjan.github.io.
Teimur Gasanov
Python/Go/Javascript full stack developer
