GPN 2024 Annual Meeting

In this tutorial, researchers and practitioners will learn to leverage Kubernetes to scale and parallelize AI/ML and advanced analytics workloads, such that they can run their Big Data workloads on any size of cluster, ranging from local to commercial clouds to publicly-funded research platforms.

This tutorial will cover building reproducible and portable containers for software using Docker and deploying them to Kubernetes clusters. Each step of the deployment from local development to Kubernetes cluster deployment will be covered, including building and pushing custom containers to image registries, building Kubernetes pod and job YAML files, and deploying pods and jobs to the cluster. Hands on examples including running ML jobs in the NSF National Research Platform.