What’s new in CSE 3.1?

For greenfield installations, please get started with CSE introduction.

1. Overview

Terminology:

CSE 3.1.3

CSE 3.1.2

CSE 3.1.1

CSE 3.1.0

User Context Diagram

user-ctx

2. Provider workflows

2.1 Compatibility matrix and relevant features

Kubernetes Container Clusters UI plugin compatibility matrix

CSE Server/CLI UI plugin Cloud Director Comments
3.1.3 3.3.0 10.3.1+ Must use UI Plugin 3.3.0. Plugin ships on VMware Cloud Director 10.3.3 Download page
3.1.3 2.2.0 10.2.2 Plugin ships with VCD
3.1.2 3.2.0 10.3.2 Plugin ships with VCD
3.1.2 3.1.0 10.3.1 Plugin ships with VCD
3.1.2 2.2.0 10.2.2 Plugin ships with VCD
3.1.2 1.0.3 10.1  
3.1.1 3.1.0 10.3.1 Plugin ships with VCD
3.1.1 2.2.0 10.2.2 Plugin ships with VCD
3.1.1 1.0.3 10.1  
3.1.0 3.0.0 10.3.0 Plugin ships with VCD
3.1.0 2.0.0 10.2.0 Plugin ships with VCD
3.1.0 1.0.3 10.1  

Native cluster compatiblity matrix

CSE Server/CLI Cloud Director NSX-T NSX-V Comments
3.1.3 10.3.1+ 3.1.1 6.4.10† Cluster representation as RDE 2.1.0
3.1.3 10.2.2 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.2 10.3.2 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.2 10.3.1 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.2 10.2.2 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.2 10.1 n/a 6.4.8† Cluster representation as vApp
3.1.1 10.3.1 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.1 10.2.2 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.1 10.1 n/a 6.4.8† Cluster representation as vApp
3.1.0 10.3.0 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.0 10.2.0 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.0 10.1 n/a 6.4.8† Cluster representation as vApp

† - With NSX-V, CSE doesn’t support creation of clusters on routed Org VDC networks.

TKG compatibility

TKGs compatibility matrix

CSE CLI UI plugin Cloud Director
3.1.3 3.3.0 10.3.3
3.1.2 3.2.0 10.3.2
3.1.2 3.1.0 10.3.1
3.1.2 2.2.0 10.2.2
3.1.1 3.1.0 10.3.1
3.1.1 2.2.0 10.2.2
3.1.0 3.0.0 10.3.0
3.1.0 2.0.0 10.2.0

Note : TKGs cluster management doesn’t need CSE server to be running.

TKGi compatibility matrix

CSE Server Cloud Director TKGi NSX-T
3.1.2, 3.1.1, 3.1.0 10.3.1, 10.2.2, 10.1 1.7 2.5.1

2.2 CSE Server

2.2.1 Changes in the configuration file

Refer to the sample config file

CSE 3.1.3

  1. Addition of new extra_options fields: cpi_version, csi_version, and antrea_version

CSE 3.1.2

  1. Addition of new extra_options section that allows Providers to specify proxy details for TKG clusters.

CSE 3.1.1

  1. Addition of property no_vc_communication_mode under service section.
  2. vcs section is no longer required if no_vc_communication_mode is set to True.
  3. Removal of properties default_template_name and default_template_revision from broker section.

CSE 3.1.0

  1. Removal of property api_version
  2. Addition of property legacy_mode
  3. New template cookbook 2.0 is introduced; refer to the remote_template_cookbook_url for the location
  4. mqtt property must be enabled when CSE 3.1 is configured with VCD 10.3.

2.2.2 Greenfield installation

Refer to CSE 3.1 installation.

2.2.3 Brownfield upgrade

3.1.3 CSE can be upgraded from version 3.1.2, 3.1.1, 3.1.0 and 3.0.z to version 3.1.3 GA. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.3.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then please consider upgrading to CSE 3.1.3 following these steps.

Refer to CSE 3.1 upgrade command for details.

3.1.2 CSE can be upgraded from version 3.1.1, 3.1.0 and 3.0.z to version 3.1.2 GA. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.2.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then the steps mentioned below must be followed in order to upgrade to CSE 3.1.2.

  1. Evaluate your environment for any stateful applications that run on TKG clusters powered by CSE 3.0.z. If you wish to retain these application’s data, then leverage a Kubernetes application backup/restore strategy to backup the applications data so you can restore it later.
  2. Remove tkgm from CSE 3.0.z, following these steps
  3. Upgrade CSE via cse upgrade command.
  4. Create new TKG clusters from Ubuntu 20.04 TKG OVAs, using CSE 3.1.2.
  5. Restore applications on newly created TKG clusters.

Refer to CSE 3.1 upgrade command for details.

3.1.1 CSE can be upgraded from CSE 3.1.0 and CSE 3.0.z to 3.1.1 GA. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.1.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then please consider upgrading to CSE 3.1.1 following these steps.

Refer to CSE 3.1 upgrade command for details.

3.1.0 CSE can be upgraded to 3.1.0, only from CSE 3.0.z product line. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.0.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then please consider upgrading to CSE 3.1.1 following these steps.

Refer to CSE 3.1 upgrade command for details.

2.2.4 Removal of TKG clusters from CSE 3.0.z

  1. Delete all deployed TKG clusters across all tenants via vcd-cli or Kubernetes Container Clusters UI plugin.
  2. Disable TKG deployment on all Org VDCs via vcd cse ovdc disable.
  3. Stop the CSE server.
  4. Delete all TKG templates via VCD UI.
  5. Remove the VM Placement Policy cse---tkgm from the system via VCD UI or VCD REST api.
  6. Revert CSE configuration file to disable TKG.

2.2.5 Tenant onboarding

The provider needs to perform below operations to enable Kubernetes cluster deployments in tenant organizations and tenant virtual data centers.

  1. Grant rights to the tenant users. Refer to CSE 3.1 RBAC for more details.
  2. Enable the desired organization virtual datacenter(s) for Native, TKGs, and/or TKGi deployments.
  3. Publish Kubernetes Container Clusters UI plugin to the desired organizations.

2.3 Kubernetes Container Clusters UI plugin

Provider can publish it to the desired tenants to offer Kubernetes services. Refer to publish Kubernetes Container Clusters UI plugin. For VCD 10.2.z that inter-operates with CSE 3.1.z, Provider must use the Kubernetes Container Clusters UI plugin 2.y.z that comes with VCD to offer Kubernetes Services. For VCD versions prior to 10.2 that inter-operate with CSE 3.1.z, Kubernetes Container Clusters UI plugin 1.0.3 must be installed separately by Provider and published to the desired tenants. Refer to Register CSE UI plugin 1.0.3 for more details.

3. Tenant workflows

Tenant users can manage the Kubernetes cluster deployments either through CSE CLI or Kubernetes Container Clusters UI plugin

3.1 CLI for Container Extension

CSE 3.1.0 introduces below changes in CLI

  1. Cluster upgrades (Native and vSphere with Tanzu) can now be performed using vcd cse cluster apply <upgrade_cluster.yaml>. Refer to cluster apply usage for more details.
  2. Clusters can be shared to other users using new vcd cse cluster share command. Refer to cluster share usage

3.2 Kubernetes Container Clusters UI plugin

4. FAQ

Refer to Troubleshooting and Known issues pages.