**This is an old revision of the document!**
Set up flatcar-linux
flatcar Linux is a minimal Linux distribution tailored to running container workloads. It ships with docker as well as containerd (alone).
Before booting the VM
Upload a version of flatcar_production_vmware_ova_image.vmdk to one of the VM hosts. Instead of current it might be better to upload a previous stable release and check that updating works. For example 3033.2.0 Remove any disks. Add the flatcar_production_vmware_ova_image.vmdk as the first disk (system, sda) and add a second thin provisionen 230 GB disk (container ephemeral, sdb).
You will find a flatcar_production_vmware_ova_image.vmdk.bz2. Use the filter box. Download that and unpack it. It is still in a special format not immedeiately usable by a ESXi host. After uploading to one VM host you have to copy the VMDK using vCenter to another host/data storage. The resulting copy is usable.
You can also copy the base vmdk of a snapshot of an already existing machine. Using the snapshot technique this is never touched. Note however that this of course does not get any updates so it might be really old and missing ignition configuration options found in the current docs. You can run an update in the running VM as the first task.
Per VM settings like networking can be passed to the VM using VMWare's guestinfo.ignition.* properties. These settings can be found in vCenter edit -> the VM Options -> Advanced -> Edit Configuration...
| Config key | Value |
|---|---|
| guestinfo.ignition.config.data.encoding | gz+base64 |
| guestinfo.ignition.config.data | <see below> |
passwd: users: - name: core ssh_authorized_keys: - ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFBjtMTXcKtVlorGkFwpCs1XmHakFwk3ggShgmHZyJaAhjR81bAFFZVGURBIXUnmi4DpcAV1GaNDNdw0DJvPqRo= Omar Yubikey 5 nano - ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAe26li9Rzcf+iBRftzE3WDddlDI8+HpCm+GFdep5tcWC6Q2Hzbi1GSqFTBSvpKAJRbyq49OkA3gd16ugmzImDU= Omar Yubikey 5 NFC #kernel_arguments: # should_not_exist: # - flatcar.autologin locksmith: reboot_strategy: "reboot" window_start: Thu 04:00 window_length: 1h networkd: units: # change IP and MAC addresses for other nodes! - name: 10-ens1.network contents: | [Match] MACAddress=<MAC address from vCenter kubernetes_public> [Network] Address=10.3.6.xx/24 Gateway=10.3.6.1 DNS=192.168.1.250 Domains=servants.priv NTP=192.168.1.250 - name: 10-ens2.network contents: | [Match] MACAddress=MAC address from vCenter kubernetes_internal> [Network] Address=10.6.16.xx/24 storage: filesystems: - name: oem mount: device: /dev/disk/by-label/OEM format: btrfs label: OEM - name: cont-ephemer mount: device: /dev/sdb format: xfs wipe_filesystem: true label: cont-ephemer files: - path: /grub.cfg filesystem: oem mode: 0644 contents: inline: | set oem_id="vmware" # or something else like set linux_append="" - path: /etc/hostname filesystem: root mode: 0644 contents: inline: acdh-clusterx systemd: units: # Ensure docker starts automatically instead of being socket-activated - name: docker.socket enable: false - name: update-engine.service enable: true - name: locksmithd.service enable: true - name: systemd-timesyncd.service enable: true # Mount a separate disk for all docker data - name: var-lib-docker.mount # Mount a separate disk for all docker data - name: var-lib-rancher.mount enable: true contents: | [Unit] Description=Mount ephemeral to /var/lib/rancher Before=local-fs.target [Mount] What=/dev/disk/by-label/cont-ephemer Where=/var/lib/rancher Type=xfs [Install] WantedBy=local-fs.target - name: docker.service enable: false
This confioguration specified as YAML file has to be transpiled:
cat acdh-clusterx.yaml | docker run --rm -i ghcr.io/flatcar/ct:latest | gzip -c9 | base64 -w0 > acdh-clusterx.json+gz+base64.txt
Note: After configuring the guestinfo parameters, it is probably best to create a snapshot in ESXi (or vCenter, name it initial or Leer) and then boot. Guestinfo parameters are part of the snapshot! This configruation usually runs only once in an empty, unprovisioned VM. Like this we can easily go back to the initial empty system drive that is then set up using the guestinfo.ignition.config.data. When you start from scratch change the data after reverting to the snapshot. guestinfo.ignition.config.data is part of a snapshot as are the “UEFI” settings like boot order. If these are wrong you have to change them again.
Provisioning can be forced to (re)run using a boot parameter or a magic file touch /boot/flatcar/first_boot. Note: with the above settings that is destructive! The container ephemeral volume will be recreated.
The system boots with autologin for the first time. Reboot the system once until it only shows the correct IP addresses and a login prompt (there is no valid password for core).