Deploying TensorFlow Models on Flask Part 1 - Set up trained model from TensorFlow Hub

In this tutorial series we will learn how to use trained and ready to deploy machine learning models available in TensorFlow Hub. We will than create a Flask web application for deploying the model. We will utilize trained embedding model from TensorFlow Hub and build a application for sentence similarity.

This is a multi part tutorials series, we will cover end-to-end process in below parts

Part 1: Download,set up and test trained model from TensorFlow Hub

  • Navigate to TensorFlow Hub
  • Select tf2-preview-nnlmcollection
  • Download nnlm-en-dim50, for English language.
  • Create a new directory flask-ml and open it with Visual Studio Code.
  • Copy the downloaded model inside flask-ml directory and extract files twice first it will extract tf2-preview_nnlm-en-dim50_1.tar from tf2-preview_nnlm-en-dim50_1.tar.gz and than actual files from tf2-preview_nnlm-en-dim50_1.tar.
  • Create a new directory pre_trained_model inside flask-mland move "assests, variables, saved_model.pb" to pre_trained_model.
  • After this you should be having below directory structure.

    
    └───flask-ml
        │   tf2-preview_nnlm-en-dim50_1.tar
        │   tf2-preview_nnlm-en-dim50_1.tar.gz
        │
        └───pre_trained_model
            │   saved_model.pb
            │
            ├───assets
            │       tokens.txt
            │
            └───variables
                    variables.data-00000-of-00001
                    variables.index
    
    
    Generate embeddings with model and calculate cosine similiarity

    For executing below code TensorFlow2.0 or later version is required, if you don't have tensorflow installed, run pip install tensorflow. Create "test.py" inside flask-ml and write below code. Lets understand code snippet.

    • First two lines are importing required libraries.
    • Next we are loading the downloaded model with hub.load
    • After this we are creating embedding by passing sentences to the embed
    • And finally we are calculating cosine similarity, for more information on cosine similarity refer Keras documentation. For this article it is suffice to know that cosine value near to "-1" indicates greater similarity while value near to "1" indicates higher dissimilarity

    
    import tensorflow as tf
    import tensorflow_hub as hub
    
    embed = hub.load("./pre_trained_model")
    embeddings = embed(["cat is on the floor", "dog is on the floor"])
    print(embeddings)
    
    sim = tf.keras.losses.cosine_similarity(embeddings[0], embeddings[1], axis=0)
    print(sim)
    print(sim.numpy())
    

    If you are getting output similar to as shown below than the set up is working fine

    
    
      (tf2) C:\Users\vikas\Desktop\react_tuts\New folder\flask-ml>C:/ProgramData/Anaconda3/envs/tf2/python.exe "c:/Users/vikas/Desktop/react_tuts/New folder/flask-ml/test.py"
      2020-10-26 13:36:20.132956: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
      tf.Tensor(
      [[ 0.16589954  0.0254965   0.1574857   0.17688066  0.02911299 -0.03092718
         0.19445257 -0.05709129 -0.08631689 -0.04391516  0.13032274  0.10905275
        -0.08515751  0.01056632 -0.17220995 -0.17925954  0.19556305  0.0802278
        -0.03247919 -0.49176937 -0.07767699 -0.03160921 -0.13952136  0.05959712
         0.06858718  0.22386682 -0.16653948  0.19412343 -0.05491862  0.10997339
        -0.15811177 -0.02576607 -0.07910853 -0.258499   -0.04206644 -0.20052543
         0.1705603  -0.15314153  0.0039225  -0.28694248  0.02468278  0.11069503
         0.03733957  0.01433943 -0.11048374  0.11931834 -0.11552787 -0.11110869
         0.02384969 -0.07074881]
       [ 0.22075905  0.05648435  0.12771143  0.11615398  0.06471531  0.02444898
         0.13712886 -0.06288064 -0.08560256 -0.07659081  0.06722884  0.08384311
        -0.06657998 -0.03186746 -0.12207588 -0.19775932  0.13974944  0.03533671
        -0.07951755 -0.45845005 -0.0685077   0.08533238 -0.12752111  0.11610293
         0.12298352  0.16091679 -0.16929056  0.15839423 -0.07143378  0.09375338
        -0.17743316 -0.0500968  -0.06825224 -0.21238105 -0.07613859 -0.19069687
         0.1318697  -0.07961286  0.01077813 -0.3535859   0.00888554  0.13555478
         0.04420736  0.08488994 -0.01886176  0.17098431 -0.11895745 -0.10908322
         0.01940353 -0.00427027]], shape=(2, 50), dtype=float32)
      tf.Tensor(-0.95126975, shape=(), dtype=float32)
      -0.95126975
    

    In the next part 2 we will set up Flask application for serving ML model

    Next > Set up Flask application

    Categories: TensorFlow