Artificial Intelligence – Deploying for Online Learning on GCP

Deploying a TensorFlow SavedModel to Google Cloud Platform (GCP) lets us run predictions online, at scale, without managing servers directly.

Your TensorFlow model must be stored in a Google Cloud Storage bucket. This includes the saved_model.pb file.

Step 1: Create a Deployed Model Object

First, create the model container on GCP:

gcloud ml-engine models create "deployed_classifier"
  

This prepares a container where future versions will live.

Step 2: Tell GCP Where the SavedModel Files Are

Set the environment variable that points to the model binaries:

DEPLOYMENT_SOURCE="gs://classifier_bucket123/classifier_model/binaries"
  

Step 3: Deploy the Model Version

Deploy a version of your model to Google Cloud ML Engine:

gcloud ml-engine versions create "version1" \
    --model "deployed_classifier" \
    --origin $DEPLOYMENT_SOURCE \
    --runtime-version 1.9
  

The deployment process may take a few minutes.

Step 4: Prepare Data for Prediction

Define the variables used for prediction:

MODEL_NAME="deployed_classifier"
INPUT_DATA_FILE="test.json"
VERSION_NAME="version1"
  

The test.json file contains a sample JSON input.

Step 5: Request a Prediction

Send a prediction request to your deployed model:

gcloud ml-engine predict \
    --model $MODEL_NAME \
    --version $VERSION_NAME \
    --json-instances $INPUT_DATA_FILE
  

You’ll receive JSON output containing the prediction probabilities.

Success! Your Model Is Live □

Your deployed model can now be called from apps, websites, backend services, or automated systems.

Training on GCP

Google has made the deep learning training and deployment process streamlined by allowing us to train models, store them, and deploy them with minimal code. Before training in the cloud, we should first train the model locally to confirm everything works as intended.

To begin, we must set some environment variables and organize our project files into the correct structure required by Cloud ML. Inside the training folder in this chapter’s GitHub repository, you will find the correct file layout. The __init__.py file simply tells GCP that our folder contains an executable Python package.

Next, define the job directory and training data path:

JOB_DIR="model_folder"
TRAIN_DATA="/path/to/data"
  

Now train the model locally using Cloud ML’s local runner:

gcloud ml-engine local train \
    --job-dir $JOB_DIR \
    --module-name trainer.task \
    --package-path trainer/ \
    -- --train-file $TRAIN_DATA
  

If your model trains successfully locally, you are ready for cloud training.

Cloud Training Setup

Training on Cloud ML is submitted as a job. The platform automatically sets up the environment, runs the training, and cleans up afterward.

Each training job must have a unique name. We will call our job classifier_model.

Instead of a local job directory, we now point Cloud ML to a folder inside our Google Cloud Storage bucket.

Choose the closest region to your location in the United States:

• us-west1
• us-central1
• us-east1

JOB_NAME="classifier_model"
JOB_DIR="gs://classifier_bucket123/$JOB_NAME"
PACKAGE_STAGING_PATH="gs://classifier_bucket123"
MAIN_TRAINER_MODULE="trainer.simple_classifier"
REGION="us-east1"
TRAIN_DATA="gs://classifier_bucket123/creditcard.csv"
  

Submit the Cloud Training Job

Send the job to Cloud ML with:

gcloud ml-engine jobs submit training $JOB_NAME \
    --staging-bucket $PACKAGE_STAGING_PATH \
    --job-dir $JOB_DIR \
    --package-path /trainer \
    --module-name $MAIN_TRAINER_MODULE \
    --runtime-version 1.8 \
    --region $REGION \
    -- --train-file $TRAIN_DATA
  

The --runtime-version must match your installed version of TensorFlow. You can check your version by running:

python -c "import tensorflow as tf; print(tf.__version__)"
  

Monitor Training Progress

After submitting the job, Cloud ML will confirm that training has started. You can log in to the GCP console and use the Jobs section to check:

• Training metrics
• Error messages
• Output logs
• Resource usage

This information provides the same visibility you'd normally see when training locally.

When training is complete, the script will output the SavedModel binaries to your Google Cloud Storage bucket. If you have reached this stage successfully, you are ready to deploy the model as an API.