Deploying TensorFlow ML model using Flask for Beginners

This tutorial explains how to deploy a machine learning model using Flask web framework.In this tutorial we will cover following topics to deploy a sentence similarity model using Flask.

  • Downloading and setting up TensorFlow pre-trained model from TensorFlow Hub
  • Generating embeddings with pre-trained TensorFlow model and calculate cosine similarity
  • Setting up Flask Application
  • Integrating TensorFlow model with Flask Application

Before diving deep into coding and set up lets have a look to some the terms used above.

What is TensorFlow Hub

TensorFlow Hub is a repository of trained machine learning models ready for fine-tuning and deployable anywhere. TensorFlow Hub provides many pre-trained model, in this tutorial we will use a pre-trained model to generate sentence embeddings. Checkout the models provided by TensorFlow hub here.

What is cosine similarity

Cosine similarity measures the similarity between vectors by calculating the cosine angle between the two vectors. TensorFlow provides tf.keras.losses.cosine_similarity function to compute cosine similarity between labels and predictions.Cosine similarity is a number number between -1 and 1.Cosine values closer to -1 indicate greater similarity and values closer to 1 indicate greater dissimilarity.

What is Flask Framework

Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries.Flask supports extensions that can add application features as if they were implemented in Flask itself.

Now we know about the tools and framework used in this tutorial, lets gets started with the coding part.

Deploying ML model with Flask

Install required libraries
   
  pip install tensorflow
  pip install tensorflow-hub
  pip install flask
   

Using pre-trained model from TensorFlow Hub

1. Download,set up and test pre-trained model from TensorFlow Hub.

  • Download nnlm-en-dim50 model for English language from here
  • Create a new directory flask-ml and open it with Visual Studio Code.
  • Copy the downloaded model inside flask-ml directory and extract the zip file.
  • Create a new directory pre_trained_model inside flask-ml and move "assets, variables, saved_model.pb" to pre_trained_model.

After this you should be having below directory structure.

dir_structure_1

2. Generate embeddings with downloaded model and calculate cosine similarity

Create "test.py" inside flask-ml and write below code.


  import tensorflow as tf
  import tensorflow_hub as hub
  
  embed = hub.load("./pre_trained_model")
  embeddings = embed(["cat is on the floor", "dog is on the floor"])
  print(embeddings)
  
  sim = tf.keras.losses.cosine_similarity(embeddings[0], embeddings[1], axis=0)
  print(sim)
  print(sim.numpy())
        
Lets understand the above code snippet.
  • First two lines are importing required libraries.
  • Next we are loading the downloaded model with hub.load
  • After this we are creating embedding by passing sentences to the embed
  • And finally we are calculating cosine similarity, for more information on cosine similarity refer Keras documentation. For this article it is suffice to know that cosine value near to "-1" indicates greater similarity while value near to "1" indicates higher dissimilarity

Execute above code snippet in test.py, if you are getting output similar to as shown below than the set up is working fine. Now as our model set up is working fine we can delete the test.py file.


    
  tf.Tensor(
  [[ 0.16589954  0.0254965   0.1574857   0.17688066  0.02911299 -0.03092718
      0.19445257 -0.05709129 -0.08631689 -0.04391516  0.13032274  0.10905275
      -0.08515751  0.01056632 -0.17220995 -0.17925954  0.19556305  0.0802278
      -0.03247919 -0.49176937 -0.07767699 -0.03160921 -0.13952136  0.05959712
      0.06858718  0.22386682 -0.16653948  0.19412343 -0.05491862  0.10997339
      -0.15811177 -0.02576607 -0.07910853 -0.258499   -0.04206644 -0.20052543
      0.1705603  -0.15314153  0.0039225  -0.28694248  0.02468278  0.11069503
      0.03733957  0.01433943 -0.11048374  0.11931834 -0.11552787 -0.11110869
      0.02384969 -0.07074881]
      [ 0.22075905  0.05648435  0.12771143  0.11615398  0.06471531  0.02444898
      0.13712886 -0.06288064 -0.08560256 -0.07659081  0.06722884  0.08384311
      -0.06657998 -0.03186746 -0.12207588 -0.19775932  0.13974944  0.03533671
      -0.07951755 -0.45845005 -0.0685077   0.08533238 -0.12752111  0.11610293
      0.12298352  0.16091679 -0.16929056  0.15839423 -0.07143378  0.09375338
      -0.17743316 -0.0500968  -0.06825224 -0.21238105 -0.07613859 -0.19069687
      0.1318697  -0.07961286  0.01077813 -0.3535859   0.00888554  0.13555478
      0.04420736  0.08488994 -0.01886176  0.17098431 -0.11895745 -0.10908322
      0.01940353 -0.00427027]], shape=(2, 50), dtype=float32)
  tf.Tensor(-0.95126975, shape=(), dtype=float32)
  -0.95126975
      

Setting up Flask application

1. Create app.py file

Create a new file app.py inside root directory flask-ml and write below code in app.py.

      
  from flask import Flask, render_template, request, render_template_string
  
  app = Flask(__name__)
  
  @app.route("/")
  def index():
      return render_template_string("Hello from flask")
  
  if __name__ == "__main__":
      app.run(debug=True)
      
      

You should see "Hello from flask" in browser on navigating to http://127.0.0.1:5000/

2. Develop front-end using Bootstrap, HTML and CSS

  1. Create two new directories templates and static inside flask-ml.
  2. In templates directory create a file index.html
  3. In static directory create css directory and in css create a file main.css

2. Verify directory structure

After these steps you should have directory structure as show below

dir_structure_2

3. Prepare index.html

Write below code inside index.html

        
    <!DOCTYPE html>
    <html lang="en" dir="ltr">
    
    <head>
    <!-- <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap.min.css" rel="stylesheet" id="bootstrap-css"> -->
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css"
        integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
    <link href="{{ url_for('static', filename='css/main.css') }}" rel="stylesheet">
    </head>
    
    <body>
    <div class="container">
        <div class="row">
            <div class="col-lg-6">
                <div class="form_main">
                    <h4 class="heading"><strong>Sentence </strong> Similarity <span></span></h4>
                    <div class="form">
                        <form action="{{ url_for('index', ) }}" method="post" id="contactFrm" name="contactFrm">
                            <input type="text" required="" placeholder="First Sentence" value="" name="sentence-1"
                                class="txt">
                            <input type="text" required="" placeholder="Second Sentence" value="" name="sentence-2"
                                class="txt">
                            <input type="submit" value="Check Similarity" name="Sumbit" class="txt2">
                        </form>
                    </div>
                </div>
            </div>
        </div>
    </div>
    <script src="https://code.jquery.com/jquery-3.4.1.slim.min.js"
        integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous">
    </script>
    <script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js"
        integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous">
    </script>
    <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js"
        integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous">
    </script>
    </body>  
    </html>
        
        

4. Prepare main.css

Write below code inside main.css

        
  form_main {
  width: 100%;
  }
  .form_main h4 {
  font-family: roboto;
  font-size: 20px;
  font-weight: 300;
  margin-bottom: 15px;
  margin-top: 20px;
  text-transform: uppercase;
  }
  .heading {
  border-bottom: 1px solid #fcab0e;
  padding-bottom: 9px;
  position: relative;
  }
  .heading span {
  background: #9e6600 none repeat scroll 0 0;
  bottom: -2px;
  height: 3px;
  left: 0;
  position: absolute;
  width: 75px;
  }
  .form {
  border-radius: 7px;
  padding: 6px;
  }
  .txt[type="text"] {
  border: 1px solid #ccc;
  margin: 10px 0;
  padding: 10px 0 10px 5px;
  width: 100%;
  }
  .txt_3[type="text"] {
  margin: 10px 0 0;
  padding: 10px 0 10px 5px;
  width: 100%;
  }
  .txt2[type="submit"] {
  background: #242424 none repeat scroll 0 0;
  border: 1px solid #4f5c04;
  border-radius: 25px;
  color: #fff;
  font-size: 16px;
  font-style: normal;
  line-height: 35px;
  margin: 10px 0;
  padding: 0;
  text-transform: uppercase;
  width: 30%;
  }
  .txt2:hover {
  background: rgba(0, 0, 0, 0) none repeat scroll 0 0;
  color: #5793ef;
  transition: all 0.5s ease 0s;
  }
  
  .notice {
  padding: 15px;
  background-color: #fafafa;
  border-left: 6px solid #7f7f84;
  margin-bottom: 10px;
  -webkit-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
      -moz-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
          box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
  }
  .notice-sm {
  padding: 10px;
  font-size: 80%;
  }
  .notice-lg {
  padding: 35px;
  font-size: large;
  }
  .notice-success {
  border-color: #80D651;
  }
  .notice-success>strong {
  color: #80D651;
  }
  .notice-info {
  border-color: #45ABCD;
  }
  .notice-info>strong {
  color: #45ABCD;
  }
  .notice-warning {
  border-color: #FEAF20;
  }
  .notice-warning>strong {
  color: #FEAF20;
  }
  .notice-danger {
  border-color: #d73814;
  }
  .notice-danger>strong {
  color: #d73814;
  }
        
        

5. Update app.py

Change the code in app.py as shown below

        
  from flask import Flask, render_template, request, render_template_string
  
  app = Flask(__name__)
  
  
  @app.route("/")
  def index():
      return render_template("index.html")
  
  
  if __name__ == "__main__":
      app.run(debug=True)
        
        
6. Test the flask application set up.

Run the app.py and you should see below output in browser.

flask_tf

Integrating TensorFlow model with Flask app

1. Create utils.py file

Create a file utils.py inside flask-ml and write below code in utils.py

      
  import tensorflow as tf
  import tensorflow_hub as hub
  
  
  def similarity_value(s1, s2):
      embed = hub.load("./pre_trained_model")
      embeddings = embed([s1, s2])
      sim = tf.keras.losses.cosine_similarity(
          embeddings[0], embeddings[1], axis=0)
  
      print(sim.numpy())
  
      if sim.numpy() <= -0.6:
          msg = f"Cosine Similarity is {sim.numpy()}, this indicates high similarity"
      elif sim.numpy() >= 0.6:
          msg = f"Cosine Similarity is {sim.numpy()}, this indicates high dissimilarity"
      else:
          msg = f"Cosine Similarity is {sim.numpy()}, can't decide much with this value"
      return msg
      
      
      
2. Verify directory structure

After this step you should have directory structure as shown below

dir_structure_3
3. Update app.py

Replace code in app.py with the below code

      
  from flask import Flask, render_template, request, redirect
  import utils
  
  app = Flask(__name__)
  
  
  @app.route("/", methods=['GET', 'POST'])
  def index():
  
      if request.method == 'GET':
          return render_template("index.html")
  
      if request.method == 'POST':
          result = request.form.to_dict(flat=True)
          sentence_1 = result.get("sentence-1")
          sentence_2 = result.get("sentence-2")
          sim_msg = utils.similarity_value(sentence_1, sentence_2)
          result["sim_msg"] = sim_msg
          return render_template("index.html", result=result)
  
  
  if __name__ == "__main__":
      app.run(debug=True)
      
      
      
4. Update index.html

Replace code in index.html with the below code.

      
  <!DOCTYPE html>
  <html lang="en" dir="ltr">
  
  <head>
  
      <!-- <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap.min.css" rel="stylesheet" id="bootstrap-css"> -->
      <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css"
          integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
      <link href="{{ url_for('static', filename='css/main.css') }}" rel="stylesheet">
  
  </head>
  
  <body>
      <div class="container">
          <div class="row">
              <div class="col-lg-6">
                  <div class="form_main">
                      <h4 class="heading"><strong>Sentence </strong> Similarity <span></span></h4>
                      <div class="form">
                          <form action="{{ url_for('index', ) }}" method="post" id="contactFrm" name="contactFrm">
                              <input type="text" required="" placeholder="First Sentence" value="" name="sentence-1"
                                  class="txt">
                              <input type="text" required="" placeholder="Second Sentence" value="" name="sentence-2"
                                  class="txt">
                              <input type="submit" value="Check Similarity" name="Sumbit" class="txt2">
                          </form>
                      </div>
                  </div>
              </div>
          </div>
  
          {% if result %}
  
          <div class="row">
              <div class="col-lg-6">
                  <div class="form_main">
                      <h4 class="heading"><strong>Cosine </strong> Similarity Value<span></span></h4>
  
                      <div class="notice notice-info">
                          <strong>Sentence 1</strong> : {{result["sentence-1"]}}
                      </div>
  
                      <div class="notice notice-info">
                          <strong>Sentence 2</strong> : {{result["sentence-2"]}}
                      </div>
  
  
                      <div class="notice notice-success">
                          <strong>Similarity</strong> {{result["sim_msg"]}}
                      </div>
  
                  </div>
              </div>
          </div> {% endif %}
      </div>
      <script src="https://code.jquery.com/jquery-3.4.1.slim.min.js"
          integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous">
      </script>
      <script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js"
          integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous">
      </script>
      <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js"
          integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous">
      </script>
  
  </body>
  
  </html>
      
      
3. Test the Flask application with TensorFlow Model

Run the app.py, provide input sentences and click on CHECK SIMILARITY, you should see output similar to as shown below

flask_tf

With this we successfully deployed sentence similarity machine learning model and served using Flask application that is using Bootstrap and Jinja 2 for the front-end. Let us know your feedback in the comment section or any issues you faced while following steps mentioned in this article.


Category: TensorFlow