User Guide

• 1: Setting Up Network PXE Boot
• 2: Kickstart Support in Photon OS
• 3: Packer Examples for Photon OS
• 4: Kubernetes on Photon OS
• 4.1: Prerequisites
• 4.2: Running Kubernetes on Photon OS
• 4.2.1: System Information
• 4.2.2: Prepare the Hosts
• 4.2.3: Configure Kubernetes Services on the Master
• 4.2.4: Configure the Kubernetes services on Node
• 5: Photon NFS Utilities for Mounting Remote File Systems

1 - Setting Up Network PXE Boot

Photon OS supports the Preboot Execution Environment, or PXE, over a network connection. This document describes how to set up a PXE boot server to install Photon OS.

Server Setup

To set up a PXE server, you will need to have the following:

• A DHCP server to allow hosts to get an IP address.
• A TFTP server, which is a file transfer protocol similar to FTP with no authentication.
• Optionally, an HTTP server. The HTTP server will serve the RPMs yum repo, or you can use the official Photon OS repo on Bintray. Also, this HTTP server can be used if you want to provide a kickstart config for unattended installation.

The instructions to set up the servers assume you have an Ubuntu 14.04 machine with a static IP address of 172.16.78.134.

DHCP Setup

• Install the DHCP server:

  sudo apt-get install isc-dhcp-server

• Edit the Ethernet interface in /etc/default/isc-dhcp-server to INTERFACES="eth0"
• Edit the DHCP configuration in /etc/dhcp/dhcpd.conf to allow machines to boot and get an IP address via DHCP in the range 172.16.78.230 - 172.16.78.250, for example:

  subnet 172.16.78.0 netmask 255.255.255.0 {
    range 172.16.78.230 172.16.78.250;
    option subnet-mask 255.255.255.0;
    option routers 172.16.78.134;
    option broadcast-address 172.16.78.255;
    filename "pxelinux.0";
    next-server 172.16.78.134;
  }

• Restart the DHCP server:

  sudo service isc-dhcp-server restart

TFTP Setup

• Install the TFTP server:

  sudo apt-get install tftpd-hpa

• Enable the boot service and restart the service:

  sudo update-inetd --enable BOOT
  sudo service tftpd-hpa restart

Optional: HTTP server setup

This step is only needed if you are planning to serve the ks (kickstart) config file through this server; refer to Kickstart support for details.

• Serving your local yum repo. You can install apache http web server

sudo apt-get install apache2

Mount the Photon iso to get the RPMS repo and sample ks config file.

mkdir /mnt/photon-iso
sudo mount <photon_iso> /mnt/photon-iso/

Copy the RPMS repo.

cp -r /mnt/photon-iso/RPMS /var/www/html/

To support ks, you can copy the sample config file from the iso and edit it; refer to Kickstart support for details.

cp /mnt/photon-iso/isolinux/sample_ks.cfg /var/www/html/my_ks.cfg

PXE boot files setup

• Mount photon.iso to get Linux and initrd images:

mkdir /mnt/photon-iso
sudo mount <photon_iso> /mnt/photon-iso/

• Setting the PXE boot files:

wget https://www.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.gz
tar -xvf syslinux-6.03.tar.gz
pushd /var/lib/tftpboot
cp -r /mnt/photon-iso/isolinux/* .
cp ~/syslinux-6.03/bios/com32/elflink/ldlinux/ldlinux.c32 .
cp ~/syslinux-6.03/bios/com32/lib/libcom32.c32 .
cp ~/syslinux-6.03/bios/com32/libutil/libutil.c32 .
cp ~/syslinux-6.03/bios/com32/menu/vesamenu.c32 .
cp ~/syslinux-6.03/bios/core/pxelinux.0 .
mkdir pxelinux.cfg
mv isolinux.cfg pxelinux.cfg/default

• Update repo param to point to http yum repo; you may pass official photon bintray repo.

sed -i "s/append/append repo=http:\/\/172.16.78.134\/RPMS/g" menu.cfg
popd

• Optionally, you can add your ks config file; see Kickstart support for details.

2 - Kickstart Support in Photon OS

Photon OS works with kickstart for unattended, automated installations. The kickstart configuration file can either reside in the CD-ROM attached to the host or be served through an HTTP server.

• Kickstart Capabilities
• Permitted JSON Fields
• Sample Configuration File
• Unattended Installation Through Kickstart

Kickstart Capabilities

Photon OS supports the following configurations with kickstart:

• Setting the hostname
• Setting the password
• Setting the disk to install
• Selecting whether to install the full or the minimal version of Photon OS
• Applying a post-installation script
• Adding public keys to allow the root account to log in through SSH

Permitted JSON Fields

The following is a list of allowed JSON fields and their descriptions:

Sample Configuration File

Here is a sample kickstart configuration file:

{
    "hostname": "photon-machine",
    "password":
        {
            "crypted": false,
            "text": "changeme"
        },
    "disk": "/dev/sda",
    "partitions": [
                        {"mountpoint": "/", "size": 0, "filesystem": "ext4"},
                        {"mountpoint": "/boot", "size": 128, "filesystem": "ext4"},
                        {"mountpoint": "/root", "size": 128, "filesystem": "ext4"},
                        {"size": 128, "filesystem": "swap"}
                    ],
    "packagelist_file": "packages_minimal.json",
    "additional_packages": ["vim"],
    "postinstall": [
                		"#!/bin/sh",
                    	"echo \"Hello World\" > /etc/postinstall"
                   ],
    "public_key": "<ssh-key-here>",
    "install_linux_esx": false,
    "network": {
        "type": "dhcp"
    }    
}

##Installing Root Partition as LVM

In the kickstart file modify the partitions field to mount root partition as LVM.

For example:

"disk": "/dev/sda"
"partitions":[
                {"mountpoint": "/", "size": 0, "filesystem": "ext4", "lvm":{"vg_name":"vg1", "lv_name":"rootfs"}},
                {"mountpoint": "/boot", "size": 128, "filesystem": "ext4"},

                {"mountpoint": "/root", "size": 128, "filesystem": "ext4","lvm":{"vg_name":"vg1", "lv_name":"root"}},

                {"size": 128, "filesystem": "swap","lvm":{"vg_name":"vg2", "lv_name":"swap"}}
]

Note:

• vg_name : Volume Group Name
• lv_name : Logical Volume Name

In above example rootfs, root are logical volumes in the volume group vg1 and swap is logical volume in volume group vg2, physical volumes are part of disk /dev/sda.

Multiple disks are also supported. For example:

"disk": "/dev/sda"
"partitions":[
                {"mountpoint": "/", "size": 0, "filesystem": "ext4", "lvm":{"vg_name":"vg1", "lv_name":"rootfs"}},
                {"mountpoint": "/boot", "size": 128, "filesystem": "ext4"},

                {"disk": "/dev/sdb", "mountpoint": "/root", "size": 128, "filesystem": "ext4","lvm":{"vg_name":"vg1", "lv_name":"root"}},

                {"size": 128, "filesystem": "swap","lvm":{"vg_name":"vg1", "lv_name":"swap"}}
]

If disk name is not specified, the physical volumes will be part of the default disk: dev/sda. In above example rootfs,root and swap are logical volumes in volume group vg1, physical volumes are in the disk /dev/sdb and partitions are present in /dev/sda.

Note: Mounting /boot partition as LVM is not supported.

Unattended Installation Through Kickstart

For an unattended installation, you pass the ks=<config_file> parameter to the kernel command. To pass the config file, there are two options: by providing it on the ISO or by serving it from an HTTP server.

The syntax to pass the config-file to the kernel through the ISO takes the following form:

ks=cdrom:/<config_file_path>

Here is an example:

ks=cdrom:/isolinux/my_ks.cfg

The syntax to serve the config-file to the kernel from an HTTP server (NOTE: DO NOT use https:// here) takes the following form:

ks=http://<server>/<config_file_path>

Building an ISO with a Kickstart Config File

Here’s an example of how to add a kickstart config file to the Photon OS ISO by mounting the ISO on an Ubuntu machine and then rebuilding the ISO. The following example assumes you can adapt the sample kickstart configuration file that comes with the Photon OS ISO to your needs. You can obtain the Photon OS ISO for free from Bintray at the following URL:

https://packages.vmware.com/photon

Once you have the ISO, mount it.

mkdir /tmp/photon-iso
sudo mount photon.iso /tmp/photon-iso

Then copy the content of the ISO to a writable directory and push it into the directory stack:

mkdir /tmp/photon-ks-iso
cp -r /tmp/photon-iso/* /tmp/photon-ks-iso/
pushd /tmp/photon-ks-iso/

Next, copy the sample kickstart configuration file that comes with the Photon OS ISO and modify it to suit your needs. In the ISO, the sample kickstart config file appears in the isolinux directory and is named sample_ks.cfg. The name of the directory and the name of the file might be in all uppercase letters.

cp isolinux/sample_ks.cfg isolinux/my_ks.cfg
nano isolinux/my_ks.cfg

With a copy of the sample kickstart config file open in nano, make the changes that you want.

Now add a new item to the installation menu by modifying isolinux/menu.cfg:

cat >> isolinux/menu.cfg << EOF
label my_unattended
	menu label ^My Unattended Install
	menu default
	kernel vmlinuz
	append initrd=initrd.img root=/dev/ram0 loglevel=3 photon.media=cdrom
EOF

Finally, rebuild the ISO so that it includes your kickstart config file:

mkisofs -R -l -L -D -b isolinux/isolinux.bin -c isolinux/boot.cat \
		-no-emul-boot -boot-load-size 4 -boot-info-table -V "PHOTON_$(date +%Y%m%d)" \
		. > <new_iso_path>.iso

popd

3 - Packer Examples for Photon OS

Packer is an open source tool that enables you to create identical machine images for multiple platforms.

VMware maintains a GitHub project that includes examples for creating Photon OS machine images using Packer.

Vagrant Boxes

vmware/photon-packer-templates

This project provides examples to automate the creation of Photon OS machine images as Vagrant boxes using Packer and the Packer Plugins for VMware (vmware-iso) and Virtualbox (virtualbox).

The Vagrant boxes included in the project can be run on the following providers:

• VMware Fusion (vmware_desktop)
• VMware Workstation Pro (vmware_desktop)
• VirtualBox (virtualbox)

This project is also used to generate the offical vmware/photon Vagrant boxes.

All examples are authored in the HashiCorp Configuration Language (“HCL2”).

4 - Kubernetes on Photon OS

You can use Kubernetes with Photon OS. The instructions in this section present a manual configuration that gets one worker node running to help you understand the underlying packages, services, ports, and so forth.

The Kubernetes package provides several services: kube-apiserver, kube-scheduler, kube-controller-manager, kubelet, kube-proxy. These services are managed by systemd. Their configuration resides in a central location: /etc/kubernetes.

4.1 - Prerequisites

You need two or more machines with the 3.0 general availability or later version of Photon OS installed.

4.2 - Running Kubernetes on Photon OS

The procedure describes how to break the services up between the hosts.

The first host, photon-master, is the Kubernetes master. This host runs the kube-apiserver, kube-controller-manager, and kube-scheduler. In addition, the master also runs etcd. Although etcd is not needed on the master if etcd runs on a different host, this guide assumes that etcd and the Kubernetes master run on the same host. The remaining host, photon-node, is the node and runs kubelet, proxy, and docker.

4.2.1 - System Information

Hosts:

photon-master = 192.168.121.9
photon-node = 192.168.121.65

4.2.2 - Prepare the Hosts

The following packages should already be installed on the full version of Photon OS, but you might have to install them on the minimal version of Photon OS. If the tdnf command returns “Nothing to do,” the package is already installed.

• Install Kubernetes on all hosts–both photon-master and photon-node.

tdnf install kubernetes

• Install iptables on photon-master and photon-node:

tdnf install iptables

• Open the tcp port 8080 (api service) on the photon-master in the firewall

iptables -A INPUT -p tcp --dport 8080 -j ACCEPT

• Open the tcp port 10250 (api service) on the photon-node in the firewall

iptables -A INPUT -p tcp --dport 10250 -j ACCEPT

• Install Docker on photon-node:

tdnf install docker

• Add master and node to /etc/hosts on all machines (not needed if the hostnames are already in DNS). Make sure that communication works between photon-master and photon-node by using a utility such as ping.

echo "192.168.121.9	photon-master
192.168.121.65	photon-node" >> /etc/hosts

• Edit /etc/kubernetes/config, which will be the same on all the hosts (master and node), so that it contains the following lines:

# Comma separated list of nodes in the etcd cluster
KUBE_MASTER="--master=http://photon-master:8080"

# logging to stderr routes it to the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"

# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"

# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow_privileged=false"

4.2.3 - Configure Kubernetes Services on the Master

Perform the following steps to configure Kubernetes services on the master:

1. Edit /etc/kubernetes/apiserver to appear as such. The service_cluster_ip_range IP addresses must be an unused block of addresses, not used anywhere else. They do not need to be routed or assigned to anything.

   # The address on the local server to listen to.
   KUBE_API_ADDRESS="--address=0.0.0.0"
   
   # Comma separated list of nodes in the etcd cluster
   KUBE_ETCD_SERVERS="--etcd-servers=http://127.0.0.1:4001"
   
   # Address range to use for services
   KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
   
   # Add your own
   KUBE_API_ARGS=""
   

2. Start the appropriate services on master:

   for SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; do
   	systemctl restart $SERVICES
   	systemctl enable $SERVICES
   	systemctl status $SERVICES
   done
   

3. To add the other node, create the following node.json file on the Kubernetes master node:

   {
       "apiVersion": "v1",
       "kind": "Node",
       "metadata": {
           "name": "photon-node",
           "labels":{ "name": "photon-node-label"}
       },
       "spec": {
           "externalID": "photon-node"
       }
   }
   

4. Create a node object internally in your Kubernetes cluster by running the following command:

   $ kubectl create -f ./node.json
   
   $ kubectl get nodes
   NAME                LABELS              STATUS
   photon-node         name=photon-node-label     Unknown
   

Note: The above example only creates a representation for the node photon-node internally. It does not provision the actual photon-node. Also, it is assumed that photon-node (as specified in name) can be resolved and is reachable from the Kubernetes master node.

4.2.4 - Configure the Kubernetes services on Node

Perform the following steps to configure the kubelet on the node:

1. Edit /etc/kubernetes/kubelet to appear like this:

   ###
   # Kubernetes kubelet (node) config
   
   # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
   KUBELET_ADDRESS="--address=0.0.0.0"
   
   # You may leave this blank to use the actual hostname
   KUBELET_HOSTNAME="--hostname_override=photon-node"
   
   # location of the api-server
   KUBELET_API_SERVER="--kubeconfig=/etc/kubernetes/kubeconfig"
   
   # Add your own
   #KUBELET_ARGS=""
   

2. Edit /etc/kubernetes/kubeconfig to appear like this:

   clusters:
   - cluster:
       server: http://photon-master:8080
   

3. Start the appropriate services on the node (photon-node):

   for SERVICES in kube-proxy kubelet docker; do 
       systemctl restart $SERVICES
       systemctl enable $SERVICES
       systemctl status $SERVICES 
   done
   

4. Check to make sure that the cluster can now see the photon-node on photon-master and that its status changes to Ready.

   kubectl get nodes
   NAME                LABELS              STATUS
   photon-node          name=photon-node-label     Ready
   

   If the node status is NotReady, verify that the firewall rules are permissive for Kubernetes.

   • Deletion of nodes: To delete photon-node from your Kubernetes cluster, one should run the following on photon-master (please do not do it, it is just for information):

   kubectl delete -f ./node.json
   

Result

You should have a functional cluster. You can now launch a test pod. For an introduction to working with Kubernetes, see Kubernetes documentation.

5 - Photon NFS Utilities for Mounting Remote File Systems

This document describes how to mount a remote file system on Photon OS by using nfs-utils, a commonly used package that contains tools to work with the Network File System protocol (NFS).

Check a Remote Server

showmount  -e nfs-servername or ip

Example:

showmount -e eastern-filer.eng.vmware.com
showmount -e 10.109.87.129

Mount a Remote File System in Photon Full

The nfs-utils package is installed by default in the full version of Photon OS. Here is how to mount a directory through NFS on Photon OS:

mount -t nfs nfs-ServernameOrIp:/exportfolder /mnt/folder

Example:

mount -t nfs eastern-filer.eng.vmware.com:/export/filer /mnt/filer
mount -t nfs 10.109.87.129:/export /mnt/export

Mount a Remote File System in Photon Minimal

The nfs-utils package is not installed in the minimal version of Photon OS. You install it by running the following command:

tdnf install nfs-utils

For more information on installing packages with the tdnf command, see the Photon OS Administration Guide.

Once nfs-utils is installed, you can mount a file system by running the following commands, replacing the placeholders with the path of the directory that you want to mount:

mount nfs
mount -t nfs nfs-ServernameOrIp:/exportfolder /mnt/folder