Artificial Intelligence – Deploying for Online Learning on GCP

Deploying a TensorFlow SavedModel to Google Cloud Platform (GCP) lets us run predictions online, at scale, without managing servers directly.

Your TensorFlow model must be stored in a Google Cloud Storage bucket. This includes the saved_model.pb file.

Step 1: Create a Deployed Model Object

First, create the model container on GCP:

gcloud ml-engine models create "deployed_classifier"
  

This prepares a container where future versions will live.

Step 2: Tell GCP Where the SavedModel Files Are

Set the environment variable that points to the model binaries:

DEPLOYMENT_SOURCE="gs://classifier_bucket123/classifier_model/binaries"
  

Step 3: Deploy the Model Version

Deploy a version of your model to Google Cloud ML Engine:

gcloud ml-engine versions create "version1" \
    --model "deployed_classifier" \
    --origin $DEPLOYMENT_SOURCE \
    --runtime-version 1.9
  

The deployment process may take a few minutes.

Step 4: Prepare Data for Prediction

Define the variables used for prediction:

MODEL_NAME="deployed_classifier"
INPUT_DATA_FILE="test.json"
VERSION_NAME="version1"
  

The test.json file contains a sample JSON input.

Step 5: Request a Prediction

Send a prediction request to your deployed model:

gcloud ml-engine predict \
    --model $MODEL_NAME \
    --version $VERSION_NAME \
    --json-instances $INPUT_DATA_FILE
  

You’ll receive JSON output containing the prediction probabilities.

Success! Your Model Is Live □

Your deployed model can now be called from apps, websites, backend services, or automated systems.