What’s new in CSE 3.1?

For greenfield installations, please get started with CSE introduction.

1. Overview

Terminology:

CSE 3.1.1

CSE 3.1.0

User Context Diagram

user-ctx

2. Provider workflows

2.1 Compatibility matrix and relevant features

Kubernetes Container Clusters UI plugin compatibility matrix

CSE Server/CLI UI plugin Cloud Director Comments
3.1.1 3.1.0 10.3.1 Plugin ships with VCD
3.1.1 2.2.0 10.2.2 Plugin ships with VCD
3.1.1 1.0.3 10.1 Download from here
3.1.0 3.0.0 10.3.0 Plugin ships with VCD
3.1.0 2.0.0 10.2.0 Plugin ships with VCD
3.1.0 1.0.3 10.1 Download from here

Native cluster comaptiblity matrix

CSE Server/CLI Cloud Director NSX-T NSX-V Comments
3.1.1 10.3.1 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.1 10.2.2 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.1 10.1 n/a 6.4.8† Cluster representation as vApp
3.1.0 10.3.0 3.1.1 6.4.10† Cluster representation as RDE 2.0.0
3.1.0 10.2.0 3.1.1 6.4.10† Cluster representation as RDE 1.0.0
3.1.0 10.1 n/a 6.4.8† Cluster representation as vApp

† - With NSX-V, CSE doesn’t support creation of clusters on routed Org VDC networks.

TKG compatibility matrix

CSE Server/CLI Cloud Director NSX-T with Avi
3.1.1 10.3.1 NSX-T 3.1.3 and Avi 21.1.1

Note: Ubuntu 20.04 Kubernetes OVAs from VMware Tanzu Kubernetes Grid Versions 1.4.0, 1.3.1, 1.3.0 are supported.

TKG-S compatibility matrix

CSE CLI UI plugin Cloud Director
3.1.1 3.1.0 10.3.1
3.1.1 2.2.0 10.2.2
3.1.0 3.0.0 10.3.0
3.1.0 2.0.0 10.2.0

Note : TKG-S cluster management doesn’t need CSE server to be running.

TKG-I compatibility matrix

CSE Server Cloud Director Ent-PKS NSX-T
3.1.1, 3.1.0 10.3.1, 10.2.2, 10.1 1.7 2.5.1

2.2 CSE Server

2.2.1 Changes in the configuration file

Refer to the sample config file

CSE 3.1.1

  1. Addition of property no_vc_communication_mode under service section.
  2. vcs section is no longer required if no_vc_communication_mode is set to True.
  3. Removal of properties default_template_name and default_template_revision from broker section.

CSE 3.1.0

  1. Removal of property api_version
  2. Addition of property legacy_mode
  3. New template cookbook 2.0 is introduced; refer to the remote_template_cookbook_url for the location
  4. mqtt property must be enabled when CSE 3.1 is configured with VCD 10.3.

2.2.2 Greenfield installation

Refer to CSE 3.1 installation.

2.2.3 Brownfield upgrade

3.1.1
CSE can be upgraded from CSE 3.1.0 and CSE 3.0.z to 3.1.1 GA. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.1.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then the steps mentioned below must be followed in order to upgrade to CSE 3.1.1.

  1. Evaluate your environment for any stateful applications that run on TKG clusters powered by CSE 3.0.z. If you wish to retain these application’s data, then leverage a Kubernetes application backup/restore strategy to backup the applications data so you can restore it later.
  2. The next set of steps lead you through removal of TKG clusters from CSE 3.0.z after which you can safely upgrade to CSE 3.1.1
    2.1. Delete all deployed TKG clusters across all tenants via vcd-cli or Kubernetes Container Clusters UI plugin.
    2.2. Disable TKG deployment on all Org VDCs via vcd cse ovdc disable.
    2.3. Stop the CSE server.
    2.4. Delete all TKG templates via VCD UI.
    2.5. Remove the VM Placement Policy cse---tkgm from the system via VCD UI or VCD REST api.
    2.6. Revert CSE configuration file to disable TKG.
  3. Upgrade CSE via cse upgrade command.
  4. Create new TKG clusters from Ubuntu 20.04 TKG OVAs, using CSE 3.1.1.
  5. Restore applications on newly created TKG clusters.

Refer to CSE 3.1 upgrade command for details.

3.1.0
CSE can be upgraded to 3.1.0, only from CSE 3.0.z product line. Any CSE release older than CSE 3.0.0 first needs to be upgraded to CSE 3.0.z product line before it can be upgraded to CSE 3.1.0.

Note : If Tanzu Kubernetes Grid (TKG) distribution is enabled on CSE 3.0.z, then please consider upgrading to CSE 3.1.1 following these steps.

Refer to CSE 3.1 upgrade command for details.

2.2.4 Tenant onboarding

The provider needs to perform below operations to enable Kubernetes cluster deployments in tenant organizations and tenant virtual data centers.

  1. Grant rights to the tenant users. Refer to CSE 3.1 RBAC for more details.
  2. Enable the desired organization virtual datacenter(s) for Native, TKG-S, and/or TKG-I deployments.
  3. Publish Kubernetes Container Clusters UI plugin to the desired organizations.

2.3 Kubernetes Container Clusters UI plugin

Kubernetes Container Clusters UI plugin 3.1.0 is available out of the box with VCD 10.3.1.
Kubernetes Container Clusters UI plugin 3.0.0 is available out of the box with VCD 10.3.0.

Provider can publish it to the desired tenants to offer Kubernetes services. Refer to publish Kubernetes Container Clusters UI plugin. For VCD 10.2.z that inter-operates with CSE 3.1.z, Provider must use the Kubernetes Container Clusters UI plugin 2.y.z that comes with VCD to offer Kubernetes Services. For VCD versions prior to 10.2 that inter-operate with CSE 3.1.z, Kubernetes Container Clusters UI plugin 1.0.3 must be installed separately by Provider and published to the desired tenants. Refer to Register CSE UI plugin 1.0.3 for more details.

3. Tenant workflows

Tenant users can manage the Kubernetes cluster deployments either through CSE CLI or Kubernetes Container Clusters UI plugin

3.1 CLI for Container Extension

CSE 3.1.0 introduces below changes in CLI

  1. Cluster upgrades (Native and vSphere with Tanzu) can now be performed using vcd cse cluster apply <upgrade_cluster.yaml>. Refer to cluster apply usage for more details.
  2. Clusters can be shared to other users using new vcd cse cluster share command. Refer to cluster share usage

3.2 Kubernetes Container Clusters UI plugin

For VCD 10.3.z, you must use the Kubernetes Container Clusters UI plugin 3.y.0 that comes with VCD to manage the cluster deployments. For VCD 10.2.z, you must use the Kubernetes Container Clusters UI plugin 2.y.0 that comes with VCD to manage the cluster deployments.

If you are working with VCD versions prior to 10.2, you must use the Kubernetes Container Clusters UI plugin 1.0.3 to manage the cluster deployments.

4. FAQ

Refer to Troubleshooting and Known issues pages.