Kubernetes on Photon OS
You can use Kubernetes with Photon OS. The instructions in this section present a manual configuration that gets one worker node running to help you understand the underlying packages, services, ports, and so forth.
The Kubernetes package provides several services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd. Their configuration resides in a central location: /etc/kubernetes.
1 - Prerequisites
You need two or more machines with version 3.0 “GA” or later of Photon OS installed. It is recommended to use the latest 4.0 GA version.
2 - Running Kubernetes on Photon OS
The procedure describes how to break the services up between the hosts.
The first host, photon-master
, is the Kubernetes master. This host runs the kube-apiserver
, kube-controller-manager
, and kube-scheduler
. In addition, the master also runs etcd
. Although etcd
is not needed on the master if etcd
runs on a different host, this guide assumes that etcd
and the Kubernetes master run on the same host. The remaining host, photon-node
, is the node and runs kubelet
, proxy
, and docker
.
2.1 - System Information
Hosts:
photon-master = 192.168.121.9
photon-node = 192.168.121.65
2.2 - Prepare the Hosts
The following packages have to be installed. If the tdnf
command returns “Nothing to do,” the package is already installed.
Install Kubernetes on all hosts (both photon-master
and photon-node
).
Install iptables on photon-master and photon-node:
Open the tcp port 8080 (api service) on the photon-master in the firewall
iptables -A INPUT -p tcp --dport 8080 -j ACCEPT
Open the tcp port 10250 (api service) on the photon-node in the firewall
iptables -A INPUT -p tcp --dport 10250 -j ACCEPT
Install Docker on photon-node:
Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping.
echo "192.168.121.9 photon-master
192.168.121.65 photon-node" >> /etc/hosts
Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines:
# Comma separated list of nodes in the etcd cluster
KUBE_MASTER="--master=http://photon-master:8080"
# logging to stderr routes it to the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow_privileged=false"
2.3 - Configure Kubernetes Services on the Master
Perform the following steps to configure Kubernetes services on the master:
Edit /etc/kubernetes/apiserver
to appear as such. The service_cluster_ip_range
IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.
# The address on the local server to listen to.
KUBE_API_ADDRESS="--address=0.0.0.0"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# Add your own
KUBE_API_ARGS=""
Start the appropriate services on master:
for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done
To add the other node, create the following node.json
file on the Kubernetes master node:
{
"apiVersion": "v1",
"kind": "Node",
"metadata": {
"name": "photon-node",
"labels":{ "name": "photon-node-label"}
},
"spec": {
"externalID": "photon-node"
}
}
Create a node object internally in your Kubernetes cluster by running the following command:
$ kubectl create -f ./node.json
$ kubectl get nodes
NAME LABELS STATUS
photon-node name=photon-node-label Unknown
Note: The above example only creates a representation for the node photon-node
internally. It does not provision the actual photon-node
. Also, it is assumed that photon-node
(as specified in name
) can be resolved and is reachable from the Kubernetes master node.
2.4 - Configure the Kubernetes services on Node
Perform the following steps to configure the kubelet on the node:
Edit /etc/kubernetes/kubelet to appear like this:
###
# Kubernetes kubelet (node) config
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=0.0.0.0"
# You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname_override=photon-node"
# location of the api-server
KUBELET_API_SERVER="--kubeconfig=/etc/kubernetes/kubeconfig"
# Add your own
#KUBELET_ARGS=""
Make sure that the api-server end-point located in /etc/kubernetes/kubeconfig, targets the api-server in the master node and does not fall into the loopback interface:
apiVersion: v1
clusters:
- cluster:
server: <ip_master_node>:8080
Start the appropriate services on the node (photon-node):
for SERVICES in kube-proxy kubelet docker; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done
Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to Ready.
kubectl get nodes
NAME LABELS STATUS
photon-node name=photon-node-label Ready
If the node status is NotReady
, verify that the firewall rules are permissive for Kubernetes.
- Deletion of nodes: To delete photon-node from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information):
kubectl delete -f ./node.json
Result
You should have a functional cluster. You can now launch a test pod. For an introduction to working with Kubernetes, see Kubernetes documentation.