How can current Ubuntu versions be installed via PXE network boot and an automated installation?
Introduction
Instead of having a small package specific to the task, it's now necessary to use the regular full ISO image. Only the server installer Subiquity can be automated, so it's required to use the server image even for a desktop installation.
The following steps are for the current version 20.10 (Groovy Gorilla). For future versions, replace all occurrences of 20.10
by the version number you want to install.
Prerequisites
Prerequisites on the server machine that is going to serve the PXE files, installer image and autoinstall configuration:
- A working TFTP server and corresponding DHCP configuration
- A working web server
- Having the packages
pxelinux
,syslinux-efi
,syslinux-common
installed
Serving the ISO image and autoinstall config
- Download the full live server image:
wget https://releases.ubuntu.com/20.10/ubuntu-20.10-live-server-amd64.iso
- Copy the ISO image to your web server
- Create a directory on the web server to hold the autoinstall configuration. Create the files
meta-data
anduser-data
in the directory. - Write your autoinstall configuration to the file
user-data
by using https://ubuntu.com/server/docs/install/autoinstall-reference as a reference
Preparing and serving the PXE environment
Go to the directory your TFTP server serves, by default
/srv/tftp
Copy the ISO image downloaded in the previous section to the current directory
Create some directories that we are going to use in the following steps:
mkdir -p iso_mount init boot/uefi boot/bios pxelinux.cfg
Mount the image and extract the kernel and initrd:
mount -o loop ubuntu-20.10-live-server-amd64.iso iso_mount/ cp -p iso_mount/casper/vmlinuz init/ cp -p iso_mount/casper/initrd init/ umount iso_mount
Prepare the PXELINUX files for booting via UEFI as well as via BIOS:
# UEFI cp -p /usr/lib/SYSLINUX.EFI/efi64/syslinux.efi boot/uefi cp -p /usr/lib/syslinux/modules/efi64/ldlinux.e64 boot/uefi cp -p /usr/lib/syslinux/modules/efi64/libcom32.c32 boot/uefi cp -p /usr/lib/syslinux/modules/efi64/libutil.c32 boot/uefi cp -p /usr/lib/syslinux/modules/efi64/vesamenu.c32 boot/uefi # BIOS cp -p /usr/lib/PXELINUX/pxelinux.0 boot/bios cp -p /usr/lib/syslinux/modules/bios/ldlinux.c32 boot/bios cp -p /usr/lib/syslinux/modules/bios/libcom32.c32 boot/bios cp -p /usr/lib/syslinux/modules/bios/libutil.c32 boot/bios cp -p /usr/lib/syslinux/modules/bios/vesamenu.c32 boot/bios
Configure your DHCP server to set the boot filename to
boot/uefi/syslinux.efi
for UEFI clients andboot/bios/pxelinux.0
for BIOS clientsCreate a PXELINUX config at
pxelinux.cfg/default
similar to the following. Replace the URLs in angle brackets by the URLs to your web server: Replace<iso url>
by the URL to the ISO image and<autoinstall url>
by the URL to the directory containing the autoinstall configuration (including a trailing slash!).DEFAULT vesamenu.c32 PROMPT 0 NOESCAPE 1 MENU TITLE PXE System Installation LABEL Ubuntu 20.10 MENU LABEL ubuntu_20.10 KERNEL ../../init/vmlinuz INITRD ../../init/initrd APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=<iso url> autoinstall ds=nocloud-net;s=<autoinstall url>
Link the PXELINUX config into the
boot
directories for UEFI and BIOS:ln -s ../../pxelinux.cfg boot/uefi/pxelinux.cfg ln -s ../../pxelinux.cfg boot/bios/pxelinux.cfg
Conclusion
You should now be able to boot both UEFI and BIOS clients via PXE into the Subiquity installer, which will then run automated according to your autoinstall config.
Desktop installation
If you are installing a desktop system, it's necessary to install the ubuntu-desktop
package.
You'll also need to fix the network configuration in the installed system, since Subiquity enables systemd-networkd
instead of NetworkManager
, which is normally used on desktop systems:
Remove the network configuration created by Subiquity:
rm /etc/netplan/00-installer-config.yaml
Create a file
/etc/netplan/01-network-manager-all.yaml
with the following contents:# Let NetworkManager manage all devices on this system network: version: 2 renderer: NetworkManager
Run
netplan apply
or reboot
Well, I could just as well put in my take on this... so here it goes!
Intro
Below is a full PXE guide, from setting up server and all required services, up to first boots. It also includes instructions for both Ubuntu 20.04.1 and 20.10, with automatic install of Ubuntu Server, interactive install of Server, and Live boot of Desktop (like PXE version of "Live CD"). Guide had been modified slightly to include all options in one boot menu with all variations. Both BIOS and UEFI scenarios are explained.
I am using component of Syslinux called "lpxelinux" (first lowercase "L"), which has support to boot kernel and init from HTTP as well. syslinux.efi
already has this as well. HTTP enables faster transfers, plus enables us to have a little bit different organization of files, plus some other minor advantages down the line (for larger implementations).
This is full step-by-step guide!
PXE Server - Ubuntu 20.04.1 LTS
Install Ubuntu Server as usual, start with minimum installed options, but I usually enable OpenSSH server to enable remote administration. After install make sure to update it fully
sudo su
apt-get update && apt-get upgrade -y
Tools and requirements
We need TFTP server, HTTP(S) server, and DHCP server. You can use other servers if you already have them, but for ease of following we will install everything on our newly installed Ubuntu 20.04.1 LTS
I picked most popular packages:
apt-get install tftpd-hpa apache2 isc-dhcp-server
We will also need to fetch syslinux files. Since Marian's guide already showed how to get those from apt packages, I'll go other way, for completeness sake (you can mix and match approaches as you wish). We get files from official kernel.org
mkdir /root/pxe
mkdir /root/pxe/syslinux
cd /root/pxe/syslinux
wget https://mirrors.edge.kernel.org/pub/linux/utils/boot/syslinux/syslinux-6.03.tar.gz
tar -xf syslinux-6.03.tar.gz
cd syslinux-6.03/
ll
I used the chance to create a temp dir for us under /root/pxe
and syslinux
directory underneath it. Feel free to place these files wherever you wish, just make sure to amend commands afterwards.
SYSLINUX - LPXELINUX (files)
Next we copy files that we'll need. Feel free to copy more modules, but these are enough.
TFTPD already has a directory which we will use, under: /var/lib/tftpboot/
Copy the files there, in separate directories (some files have same names but are intended for different architectures)
# files for 64bit uefi
mkdir /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/efi/syslinux.efi /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/com32/menu/menu.c32 /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/com32/menu/vesamenu.c32 /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/com32/libutil/libutil.c32 /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/com32/elflink/ldlinux/ldlinux.e64 /var/lib/tftpboot/efi64
cp /root/pxe/syslinux/syslinux-6.03/efi64/com32/lib/libcom32.c32 /var/lib/tftpboot/efi64
cd /var/lib/tftpboot/efi64
ll
# files for 32bit bios
mkdir /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/core/lpxelinux.0 /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/com32/menu/menu.c32 /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/com32/menu/vesamenu.c32 /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/com32/libutil/libutil.c32 /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/com32/elflink/ldlinux/ldlinux.c32 /var/lib/tftpboot/bios
cp /root/pxe/syslinux/syslinux-6.03/bios/com32/lib/libcom32.c32 /var/lib/tftpboot/bios
cd /var/lib/tftpboot/bios
ll
DHCP server
DHCP server itself should have a static IP. So let's do that (if you haven't done it during install).
Edit the network config (YAML file):
nano /etc/netplan/00-installer-config.yaml
Contents:
# This is the network config written by 'subiquity'
network:
ethernets:
eth0:
addresses: [10.10.2.1/24]
gateway4: 10.10.2.99
nameservers:
addresses: [1.1.1.1, 8.8.8.8]
version: 2
Apply the config:
netplan apply
Next we need to enter basic config for DHCP server's zones and pools. I'm doing some extra config here to show you that your future PXE clients can be in subnets different from your PXE server. In my case PXE server (with all services) is at 10.10.2.1
. I have two subnets connected via virtual router : 10.10.1.0/24
and 10.10.2.0/24
nano /etc/dhcp/dhcpd.conf
Contents (just an example):
# minimal sample /etc/dhcp/dhcpd.conf
default-lease-time 600;
max-lease-time 7200;
subnet 10.10.1.0 netmask 255.255.255.0 {
range 10.10.1.110 10.10.1.120;
option routers 10.10.1.99;
option domain-name-servers 1.1.1.1, 8.8.8.8;
# option domain-name "mydomain.example";
}
subnet 10.10.2.0 netmask 255.255.255.0 {
range 10.10.2.110 10.10.2.120;
option routers 10.10.2.99;
option domain-name-servers 1.1.1.1, 8.8.8.8;
# option domain-name "mydomain.example";
}
# pxe client, uefi64
host uefi-client {
hardware ethernet fa:fa:fa:00:0e:07;
fixed-address 10.10.1.201;
next-server 10.10.2.1;
filename "efi64/syslinux.efi";
}
# pxe client, bios
host bios-client {
hardware ethernet fa:fa:fa:00:0e:17;
fixed-address 10.10.2.202;
next-server 10.10.2.1;
filename "bios/lpxelinux.0";
}
Note: Please modify the file according to your situation! You have to have correct subnets, gateways, as well as correct MAC address (hardware address) of your clients!
And you will need to set interface where DHCP deamon will listen, use ip a
to check which interface that would be for you (I have it at eth0
)
nano /etc/default/isc-dhcp-server
Contents:
INTERFACESv4="eth0"
INTERFACESv6=""
Make sure to restart the service:
systemctl restart isc-dhcp-server.service
systemctl status isc-dhcp-server.service
And enable it to autostart on server reboots:
systemctl enable isc-dhcp-server.service
If you have typos or other mistakes like duplicate IPs or client names, deamon will fail. Check logs if status is red.
tail -n 100 /var/log/syslog
OPTIONAL - router settings
If you have different subnets, make sure you add ip helper
for DHCP, also often setup as DHCP relay
in most routers. Just point it to IP address of your PXE server (if you have services spread on multiple servers, then point it to one that holds DHCP services).
TFTP server
We need to setup basic stuff for TFTP server, but only one option really needs changing/checking - path to your root TFTP directory /var/lib/tftpboot
nano /etc/default/tftpd-hpa
Contents:
# /etc/default/tftpd-hpa
TFTP_USERNAME="tftp"
#TFTP_DIRECTORY="/srv/tftp"
# /var/lib/tftpboot
TFTP_DIRECTORY="/var/lib/tftpboot"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure"
Make sure to restart the service as well.
systemctl restart tftpd-hpa
And enable it to autostart on server reboots:
systemctl enable tftpd-hpa
You can use TFTP client to test the service if you want to be sure all is well.
SYSLINUX / (l)pxelinux configs
We need some configs to boot! At minimum we need one default
file for each architecture. We place them in TFTP directory
mkdir /var/lib/tftpboot/efi64/pxelinux.cfg
mkdir /var/lib/tftpboot/bios/pxelinux.cfg
touch /var/lib/tftpboot/efi64/pxelinux.cfg/default
touch /var/lib/tftpboot/bios/pxelinux.cfg/default
nano /var/lib/tftpboot/efi64/pxelinux.cfg/default
Contents of the file:
DEFAULT menu.c32
PROMPT 0
MENU TITLE PXE Boot Menu
MENU COLOR TABMSG 37;40 #80ffffff #00000000
MENU COLOR HOTSEL 30;47 #40000000 #20ffffff
MENU COLOR SEL 30;47 #40000000 #20ffffff
MENU COLOR SCROLLBAR 30;47 #40000000 #20ffffff
MENU WIDTH 80
MENU MARGIN 22
MENU PASSWORDMARGIN 26
MENU ROWS 6
MENU TABMSGROW 15
MENU CMDLINEROW 15
MENU ENDROW 24
MENU PASSWORDROW 12
MENU TIMEOUTROW 13
MENU VSHIFT 6
NOESCAPE 1
ALLOWOPTIONS 0
MENU AUTOBOOT Starting Local System in # seconds
LABEL bootlocal
MENU LABEL ^Local Boot
MENU DEFAULT
LOCALBOOT 0
TIMEOUT 300
TOTALTIMEOUT 3000
LABEL UbuntuServer-20.04.1-auto
MENU LABEL Ubuntu 20.04.1 Live Auto Installer
KERNEL http://10.10.2.1/ubuntu-server-20.04.1/vmlinuz
INITRD http://10.10.2.1/ubuntu-server-20.04.1/initrd
APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://10.10.2.1/ubuntu-20.04.1-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://10.10.2.1/ubuntu-server-20.04.1/
LABEL UbuntuServer-20.04.1-interactive
MENU LABEL Ubuntu 20.04.1 Live Interactive Installer
KERNEL http://10.10.2.1/ubuntu-server-20.04.1/vmlinuz
INITRD http://10.10.2.1/ubuntu-server-20.04.1/initrd
APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://10.10.2.1/ubuntu-20.04.1-live-server-amd64.iso
LABEL UbuntuDesktop-20.04.1-live
MENU LABEL Ubuntu 20.04.1 Desktop Live CD
KERNEL http://10.10.2.1/ubuntu-desktop-20.04.1/vmlinuz
INITRD http://10.10.2.1/ubuntu-desktop-20.04.1/initrd
APPEND root=/dev/ram0 ramdisk_size=3000000 boot=casper ip=dhcp netboot=url url=http://10.10.2.1/ubuntu-20.04.1-desktop-amd64.iso
LABEL UbuntuServer-20.10-auto
MENU LABEL Ubuntu 20.10 Live Auto Installer
KERNEL http://10.10.2.1/ubuntu-server-20.10/vmlinuz
INITRD http://10.10.2.1/ubuntu-server-20.10/initrd
APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://10.10.2.1/ubuntu-20.10-live-server-amd64.iso autoinstall ds=nocloud-net;s=http://10.10.2.1/ubuntu-server-20.10/
LABEL UbuntuServer-20.10-interactive
MENU LABEL Ubuntu 20.10 Live Interactive Installer
KERNEL http://10.10.2.1/ubuntu-server-20.10/vmlinuz
INITRD http://10.10.2.1/ubuntu-server-20.10/initrd
APPEND root=/dev/ram0 ramdisk_size=1500000 ip=dhcp url=http://10.10.2.1/ubuntu-20.10-live-server-amd64.iso
LABEL UbuntuDesktop-20.10-live
MENU LABEL Ubuntu 20.10 Desktop Live CD
KERNEL http://10.10.2.1/ubuntu-desktop-20.10/vmlinuz
INITRD http://10.10.2.1/ubuntu-desktop-20.10/initrd
APPEND root=/dev/ram0 ramdisk_size=3000000 boot=casper ip=dhcp netboot=url url=http://10.10.2.1/ubuntu-20.10-desktop-amd64.iso
Note I am pulling both vmlinuz
and initrd
from HTTP server. Same server is hosting ISO image and our autoinstall files. We will setup this web server next, no worries.
Since default
looks same for both BIOS and UEFI, just copy it (or link it if you wish).
cp /var/lib/tftpboot/efi64/pxelinux.cfg/default /var/lib/tftpboot/bios/pxelinux.cfg/default
If you keep them separate you can change the options on them individually. Menu also accepts linking through include
option. But if you want advanced menus, read docs or few more articles.
Web server - HTTP(S)
I will describe simple HTTP setup, but you can expand that to HTTPS later. It would be recommended if your environment isn't closed, and you have passwords set in autoinstall files. For basic lab, we use HTTP.
We need to fetch the ISOs. Since I've put in both server and desktop in the menu, I will repeat this once for each ISO. I know question is about the server, but extras won't hurt you. If you want just server or desktop simply skip the other.
This is for Ubuntu Server LTS 20.04.1
wget https://releases.ubuntu.com/20.04.1/ubuntu-20.04.1-live-server-amd64.iso -O /var/www/html/ubuntu-20.04.1-live-server-amd64.iso
Extract the kernel and initramfs (vmlinuz and initrd) to subfolder of that distro/variation/version. You can have many more on your multi-boot PXE menu.
mount /var/www/html/ubuntu-20.04.1-live-server-amd64.iso /mnt/
mkdir /var/www/html/ubuntu-server-20.04.1
cp /mnt/casper/vmlinuz /var/www/html/ubuntu-server-20.04.1/
cp /mnt/casper/initrd /var/www/html/ubuntu-server-20.04.1/
umount /mnt
This is for Desktop LTS 20.04.1
wget https://releases.ubuntu.com/20.04.1/ubuntu-20.04.1-desktop-amd64.iso -O /var/www/html/ubuntu-20.04.1-desktop-amd64.iso
And again extract, note I added different name of folder
mount /var/www/html/ubuntu-20.04.1-desktop-amd64.iso /mnt/
mkdir /var/www/html/ubuntu-desktop-20.04.1
cp /mnt/casper/vmlinuz /var/www/html/ubuntu-desktop-20.04.1/
cp /mnt/casper/initrd /var/www/html/ubuntu-desktop-20.04.1/
umount /mnt
The following is for Ubuntu Server 20.10
wget https://releases.ubuntu.com/20.10/ubuntu-20.10-live-server-amd64.iso -O /var/www/html/ubuntu-20.10-live-server-amd64.iso
Extract the files, again to separate folder
mount /var/www/html/ubuntu-20.10-live-server-amd64.iso /mnt/
mkdir /var/www/html/ubuntu-server-20.10
cp /mnt/casper/vmlinuz /var/www/html/ubuntu-server-20.10/
cp /mnt/casper/initrd /var/www/html/ubuntu-server-20.10/
umount /mnt
And finally this is for Ubuntu Desktop 20.10
wget https://releases.ubuntu.com/20.10/ubuntu-20.10-desktop-amd64.iso -O /var/www/html/ubuntu-20.10-desktop-amd64.iso
Extract the files again, to separate folder once more
mount /var/www/html/ubuntu-20.10-desktop-amd64.iso /mnt/
mkdir /var/www/html/ubuntu-desktop-20.10
cp /mnt/casper/vmlinuz /var/www/html/ubuntu-desktop-20.10/
cp /mnt/casper/initrd /var/www/html/ubuntu-desktop-20.10/
umount /mnt
That should be it. Test with your browser to see if you can reach the files in correct locations. We used default Apache directory on Ubuntu to host all files, so no extra steps should be needed.
Autoinstall file for Server 20.04 and 20.10
Ubuntu docs say you can simply install one server, then find a sample file with your selections in /var/log/installer/autoinstall-user-data
file.
To test in a lab, you can use that file as your template, but it will require some modifications.
To just have a minimal config you can use code below (again YAML code, beware spacing if doing manual edits):
nano /var/www/html/ubuntu-server-20.04.1/user-data
Contents:
#cloud-config
autoinstall:
version: 1
apt:
geoip: true
preserve_sources_list: false
primary:
- arches: [amd64, i386]
uri: http://hr.archive.ubuntu.com/ubuntu
- arches: [default]
uri: http://ports.ubuntu.com/ubuntu-ports
identity: {hostname: pxe-client, password: $6$zN/uHJD1rEXD/ETf$q8CoBt3xXmBT37RslyWcpLT1za4RJR3QEtosggRKN5aZAAf6/mYbFEQO66AIPm965glBXB1DGd0Sf.oKi.Rfx/,
realname: pxe, username: pxe}
keyboard: {layout: hr, toggle: null, variant: ''}
locale: en_US
network:
network:
version: 2
ethernets:
eth0:
dhcp4: no
dhcp6: no
ssh:
allow-pw: true
install-server: true
Note: This config will setup server with hostname pxe-client
, and username and password both being pxe
. Server will have no network (DHCP disabled), so it will skip auto-update during install (for quicker testing). And it will have OpenSSH server preinstalled, with password login. Note regional stuff (hr
= Croatia, so we have Croatian mirror setup for apt, and Croatian keyboard layout), tweak them for your country. See your sample /var/log/installer/autoinstall-user-data
as a template to add your own tweaks.
Ubuntu requires meta-data
file as well. It HAS TO be empty. So just create it in same directory where you've put user-data
file
touch /var/www/html/ubuntu-server-20.04.1/meta-data
To see more options see official docs: https://ubuntu.com/server/docs/install/autoinstall-reference
For Ubuntu 20.10 everything is the same, just copy the code, or create it in correct path:
nano /var/www/html/ubuntu-server-20.10/user-data
touch /var/www/html/ubuntu-server-20.10/meta-data
Use same user-data sample, they are interchangeable between 20.04 and 20.10. Also note the double network: network:
part.. 20.04 had a bug where it requred such setup, while 20.10 doesn't have the bug, but supports this as a backward compatibility feature.
Check your files
I would recommend you to check that you have all files in place for all variations:
ll /var/www/html/ubuntu-server-20.04.1/
ll /var/www/html/ubuntu-desktop-20.04.1/
ll /var/www/html/ubuntu-server-20.10/
ll /var/www/html/ubuntu-desktop-20.10/
Server folders should have 4 files:
initrd
meta-data
user-data
vmlinuz
Desktop folders should have 2 files:
initrd
vmlinuz
If you have trouble booting, check the permissions of files listed above. If you are doing lab, just do unsafe thing and chmod
it all with full rw permissions:
chmod -R 777 /var/www/html/*
For production setup proper permissions with just read access where needed.
Testing it all
In my lab environment, as well as when starting new production setup, I test this all in VMs first. I use Hyper-V for testing (I know, kill me, with Microsoft stuff here suddenly :) ). To test UEFI boot you create Generation 2
machine, and for BIOS boot you should use Generation 1
machine. Make sure to add enough RAM to VM to hold the RAMDISK ! So if VM is testing Server with 1,5GB RAM disk, then you need 2GB RAM for VM, and for Desktop we have 3GB RAM disk so best to have 4GB RAM attached to VM. Also, make sure to attach network interfaces to correct networks, as I mentioned before, I keep a virtual router and multiple subnets, as I need to test and proof this all for deployment in quite large and complicated environments.
Anyway, once you have VM - just boot it! If you did everything correctly you should see PXE boot, VM getting IP, then starting the menu. After 30 sec it will boot from disk, but just use arrow keys to select an option. I found UEFI booting way faster in my tests, so I usually tested with UEFI, then tested BIOS VM once UEFI was where I wanted it.
Wrapping it up
Only now will I admit I'm no Linux guru. But I spent a lot of time past month and a half doing PXE boots and setting up different environments for it and with it. This is pure Ubuntu stack in this tutorial. I plan to write one a bit more Microsoft oriented, but also based on SYSLINUX with multi-boot menu, and allowing us to boot (assorted) Linux distros (in addition to Windows). All of this can be made with any DHCP/HTTP/TFTP tools, including Microsoft Windows Server 2019, with couple roles installed on it (DHCP, IIS, WDS). Likewise, you can use any other distro to act as PXE server for your Ubuntu installs. You don't need to use Apache2 either, if you are more comfortable with eg. nginx as your HTTP server.
For the end, I did all my tests with both Ubuntu 20.04 and 20.10. The whole setup is EXACTLY THE SAME for both versions. Just change names of files and ISO downloads. If you just need one of them, just skip the parts for what you don't need.
Happy hunting!
Edit: 2020-11-14 - added everything for Ubuntu 20.10, cleaned some code, and added more meaningful names for folders since we are handling 4 different OS boots now.