Overview¶

The open source components of Kubeflow provide a diverse set of tools for data processing, training, and deployment of machine learning (ML) models. Understanding how these components work together is crucial for building effective ML workflows. Jupyter Notebook, for example, offers an interactive environment for data exploration and experimentation, while TensorFlow is a powerful open source ML framework used for building and training models. Apache Spark, on the other hand, is a distributed computing framework capable of handling large-scale data processing and ML workloads. By leveraging these tools, data scientists and engineers can build robust and efficient ML workflows tailored to their specific needs. In this section, we introduce the Kubeflow components used in the Kubeflow on vSphere platform. Our goal is to help data scientists and engineers explore the various tools available and choose the most suitable components for their specific use cases.

The open source nature of Kubeflow allows developers to contribute to the platform by creating new components or extending existing ones. This improves the functionality and capabilities of the platform and ultimately benefits the entire ML community. Specifically, Kubeflow on vSphere provides several add-on features that enhance Kubeflow. In this section, we introduce these features and highlight their benefits to help data scientists and engineers leverage the full potential of this platform.

Machine Learning Operations (MLOps) and Kubeflow are closely related, as MLOps is an approach to managing the entire lifecycle of ML models, while Kubeflow is an open source platform for building, deploying, and managing learning models at scale. Kubeflow provides several MLOps capabilities, such as automated model training and deployment workflows, version control, and automated testing and monitoring. These capabilities streamline ML workflows, reduce development time, and improve model performance. In the last part of this section, we explore how Kubeflow on vSphere leverages MLOps to provide a complete solution for managing learning models at scale, from development to deployment and monitoring.