Running Kubernetes on Photon OS

1: System Information
2: Prepare the Hosts
3: Configure Kubernetes Services on the Master
4: Configure the Kubernetes services on Node

The procedure describes how to break the services up between the hosts.

The first host, photon-master, is the Kubernetes master. This host runs the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master also runs etcd. Although etcd is not needed on the master if etcd runs on a different host, this guide assumes that etcd and the Kubernetes master run on the same host. The remaining host, photon-node, is the node and runs kubelet, proxy, and docker.

1 - System Information

Hosts:

photon-master = 192.168.121.9
photon-node = 192.168.121.65

2 - Prepare the Hosts

The following packages have to be installed. If the tdnf command returns “Nothing to do,” the package is already installed.

Install Kubernetes on all hosts (both photon-master and photon-node).
```
tdnf install kubernetes
```
Install iptables on photon-master and photon-node:
```
tdnf install iptables
```
Open the tcp port 8080 (api service) on the photon-master in the firewall
```
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
```
Open the tcp port 10250 (api service) on the photon-node in the firewall
```
iptables -A INPUT -p tcp --dport 10250 -j ACCEPT
```
Install Docker on photon-node:
```
tdnf install docker
```
Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping.
```
echo "192.168.121.9	photon-master
192.168.121.65	photon-node" >> /etc/hosts
```

Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines:

# Comma separated list of nodes in the etcd cluster
KUBE_MASTER="--master=http://photon-master:8080"

# logging to stderr routes it to the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow_privileged=false"

3 - Configure Kubernetes Services on the Master

Perform the following steps to configure Kubernetes services on the master:

Edit /etc/kubernetes/apiserver to appear as such. The service_cluster_ip_range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.

# The address on the local server to listen to.
KUBE_API_ADDRESS="--address=0.0.0.0"

# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"

# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"

# Add your own
KUBE_API_ARGS=""

Start the appropriate services on master:

for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
	systemctl restart $SERVICES
	systemctl enable $SERVICES
	systemctl status $SERVICES
done

To add the other node, create the following node.json file on the Kubernetes master node:

{
    "apiVersion": "v1",
    "kind": "Node",
    "metadata": {
        "name": "photon-node",
        "labels":{ "name": "photon-node-label"}
    },
    "spec": {
        "externalID": "photon-node"
    }
}

Create a node object internally in your Kubernetes cluster by running the following command:

$ kubectl create -f ./node.json

$ kubectl get nodes
NAME                LABELS              STATUS
photon-node         name=photon-node-label     Unknown

Note: The above example only creates a representation for the node photon-node internally. It does not provision the actual photon-node. Also, it is assumed that photon-node (as specified in name) can be resolved and is reachable from the Kubernetes master node.

4 - Configure the Kubernetes services on Node

Perform the following steps to configure the kubelet on the node:

Edit /etc/kubernetes/kubelet to appear like this:

###
# Kubernetes kubelet (node) config

# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"

# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=photon-node"

# location of the api-server
KUBELET_API_SERVER="--kubeconfig=/etc/kubernetes/kubeconfig"

# Add your own
#KUBELET_ARGS=""

Make sure that the api-server end-point located in /etc/kubernetes/kubeconfig, targets the api-server in the master node and does not fall into the loopback interface:
```
apiVersion: v1
clusters:
- cluster:
    server: <ip_master_node>:8080
```

Start the appropriate services on the node (photon-node):

for SERVICES in kube-proxy kubelet docker; do 
    systemctl restart $SERVICES
    systemctl enable $SERVICES
    systemctl status $SERVICES 
done

Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to Ready.
```
kubectl get nodes
NAME                LABELS              STATUS
photon-node          name=photon-node-label     Ready
```
If the node status is NotReady, verify that the firewall rules are permissive for Kubernetes.
- Deletion of nodes: To delete photon-node from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information):
```
kubectl delete -f ./node.json
```

Result

You should have a functional cluster. You can now launch a test pod. For an introduction to working with Kubernetes, see Kubernetes documentation.