This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Kubernetes on Photon OS

You can use Kubernetes with Photon OS. The instructions in this section present a manual configuration that gets one worker node running to help you understand the underlying packages, services, ports, and so forth.

The Kubernetes package provides several services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd. Their configuration resides in a central location: /etc/kubernetes.

1 - Prerequisites

You need two or more machines with version 3.0 “GA” or later of Photon OS installed. It is recommended to use the latest 4.0 GA version.

2 - Running Kubernetes on Photon OS

The procedure describes how to break the services up between the hosts.

The first host, photon-master, is the Kubernetes master. This host runs the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master also runs etcd. Although etcd is not needed on the master if etcd runs on a different host, this guide assumes that etcd and the Kubernetes master run on the same host. The remaining host, photon-node, is the node and runs kubelet, proxy, and docker.

2.1 - System Information

Hosts:

photon-master = 192.168.121.9
photon-node = 192.168.121.65

2.2 - Prepare the Hosts

The following packages have to be installed. If the tdnf command returns “Nothing to do,” the package is already installed.

  • Install Kubernetes on all hosts (both photon-master and photon-node).

    tdnf install kubernetes
    
  • Install iptables on photon-master and photon-node:

    tdnf install iptables
    
  • Open the tcp port 8080 (api service) on the photon-master in the firewall

    iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
    
  • Open the tcp port 10250 (api service) on the photon-node in the firewall

    iptables -A INPUT -p tcp --dport 10250 -j ACCEPT
    
  • Install Docker on photon-node:

    tdnf install docker
    
  • Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping.

    echo "192.168.121.9	photon-master
    192.168.121.65	photon-node" >> /etc/hosts
    
  • Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines:

    # Comma separated list of nodes in the etcd cluster
    KUBE_MASTER="--master=http://photon-master:8080"
    
    # logging to stderr routes it to the systemd journal
    KUBE_LOGTOSTDERR="--logtostderr=true"
    
    # journal message level, 0 is debug
    KUBE_LOG_LEVEL="--v=0"
    
    # Should this cluster be allowed to run privileged docker containers
    KUBE_ALLOW_PRIV="--allow_privileged=false"
    

2.3 - Configure Kubernetes Services on the Master

Perform the following steps to configure Kubernetes services on the master:

  1. Edit /etc/kubernetes/apiserver to appear as such. The service_cluster_ip_range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.

    # The address on the local server to listen to.
    KUBE_API_ADDRESS="--address=0.0.0.0"
    
    # Comma separated list of nodes in the etcd cluster
    KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
    
    # Address range to use for services
    KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
    
    # Add your own
    KUBE_API_ARGS=""
    
  2. Start the appropriate services on master:

    for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
    	systemctl restart $SERVICES
    	systemctl enable $SERVICES
    	systemctl status $SERVICES
    done
    
  3. To add the other node, create the following node.json file on the Kubernetes master node:

    {
        "apiVersion": "v1",
        "kind": "Node",
        "metadata": {
            "name": "photon-node",
            "labels":{ "name": "photon-node-label"}
        },
        "spec": {
            "externalID": "photon-node"
        }
    }
    
  4. Create a node object internally in your Kubernetes cluster by running the following command:

    $ kubectl create -f ./node.json
    
    $ kubectl get nodes
    NAME                LABELS              STATUS
    photon-node         name=photon-node-label     Unknown
    

Note: The above example only creates a representation for the node photon-node internally. It does not provision the actual photon-node. Also, it is assumed that photon-node (as specified in name) can be resolved and is reachable from the Kubernetes master node.

2.4 - Configure the Kubernetes services on Node

Perform the following steps to configure the kubelet on the node:

  1. Edit /etc/kubernetes/kubelet to appear like this:

    ###
    # Kubernetes kubelet (node) config
    
    # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
    KUBELET_ADDRESS="--address=0.0.0.0"
    
    # You may leave this blank to use the actual hostname
    KUBELET_HOSTNAME="--hostname_override=photon-node"
    
    # location of the api-server
    KUBELET_API_SERVER="--kubeconfig=/etc/kubernetes/kubeconfig"
    
    # Add your own
    #KUBELET_ARGS=""
    
  2. Make sure that the api-server end-point located in /etc/kubernetes/kubeconfig, targets the api-server in the master node and does not fall into the loopback interface:

    apiVersion: v1
    clusters:
    - cluster:
        server: <ip_master_node>:8080
    
  3. Start the appropriate services on the node (photon-node):

    for SERVICES in kube-proxy kubelet docker; do 
        systemctl restart $SERVICES
        systemctl enable $SERVICES
        systemctl status $SERVICES 
    done
    
  4. Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to Ready.

    kubectl get nodes
    NAME                LABELS              STATUS
    photon-node          name=photon-node-label     Ready
    

    If the node status is NotReady, verify that the firewall rules are permissive for Kubernetes.

    • Deletion of nodes: To delete photon-node from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information):
    kubectl delete -f ./node.json
    

Result

You should have a functional cluster. You can now launch a test pod. For an introduction to working with Kubernetes, see Kubernetes documentation.