Member-only story
How To Use Pipeline on Google Cloud’s Vertex AI
A simple example of Vertex AI Pipeline — the foundation of Google Cloud’s support of MLOps
Vertex AI Tutorial Series
- A Step-by-Step Guide to Training a Model on Google Cloud’s Vertex AI
- A Step-by-Step Guide to Tuning a Model on Google Cloud’s Vertex AI
- How To Operationalize a Model on Google Cloud’s Vertex AI
- How To Use AutoML on Google Cloud’s Vertex AI
- How To Use BigQuery ML on Google Cloud’s Vertex AI
- How to Use Pipeline on Google Cloud’s Vertex AI (this article)
Background
This is the sixth and last article of this series. So far, we’ve trained, optimized, and deployed models on Google Cloud’s Vertex AI. We’ve also explored using AutoML and BigQuery ML. In this last article, we’ll look at Vertex AI Pipeline. Pipeline is Google Cloud’s serverless platform for machine learning workflows. When we say machine learning workflow, we mean a sequence of steps in the model development and deployment cycle, such as data preparation/validation, model training, hyperparameter tuning, model validation, and model deployment.
To adopt ML for production, we want a repeatable, verifiable, and automatic process for making any change to the production model. The process is analogous to CI/CD in DevOps, which is how the buzzword name MLOps came about. We’ll demonstrate Vertex AI Pipeline’s core capabilities of managing this process.
Just to recap, the problem we are solving is an image classification task on the CIFAR10 dataset, which contains 60,000 32x32 images of ten classes.
Launch Our First Pipeline
We’ll walk through using Kubeflow to construct a simple pipeline. Towards the end of the article, we will also show an example of a much more complex pipeline written in Tensorflow Extended. But for now, let’s try out a very simple task: We’ll launch a pipeline of one step to train an AutoML model. The Kubeflow component of invoking AutoML is already included in the public library, which makes our example even simpler.
