How To Operationalize a Model on Google Cloud’s Vertex AI

Serving the model on Vertex AI to produce online prediction and explanation

Vertex AI (Source: Google Cloud)

Vertex AI Tutorial Series

1. A Step-by-Step Guide to Training a Model on Google Cloud’s Vertex AI
2. A Step-by-Step Guide to Tuning a Model on Google Cloud’s Vertex AI
3. How To Operationalize a Model on Google Cloud’s Vertex AI (this article)
4. How To Use AutoML on Google Cloud’s Vertex AI
5. How To Use BigQuery ML on Google Cloud’s Vertex AI
6. How to Use Pipeline on Google Cloud’s Vertex AI

Background

This is the third episode of the Vertex AI tutorial series. In the first article, we trained our first model. In the second article, we optimized it. Now it’s time to make use of it. In this article, we’ll deploy the model on Vertex AI and query it for prediction and explanation.

Just to recap, the problem we’re solving is an image classification problem. The model is trained on the CIFAR10 dataset, which contains 60,000 32x32 images of ten classes. In the previous articles, we trained and exported our best model to a Google Cloud Storage folder.

Model Deployment

The first step in the model deployment is to import the model to Vertex AI. This may sound slightly confusing. Even though the model is trained on Vertex AI and saved to Google Cloud Storage, it’s not officially in Vertex AI until we import it as a model resource. Use the following command to import the model. Check out the documentation for other available pre-built container images.

gcloud beta ai models upload --region=us-central1 \
--display-name=e2e-tutorial-model \
--container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-3:latest \
--artifact-uri=GCS_PATH_FOR_SAVED_MODEL

Next, let’s create an endpoint in Vertex AI. The endpoint will be the frontend of the model.

gcloud beta ai endpoints create --region=us-central1 \
--display-name=e2e-tutorial-ep

We need to find the model and endpoint IDs for subsequent commands. The easiest way…