Fetch autoinstall based on MAC

Question

Short:

I am looking for a way to PXE boot Ubuntu 20.04, and use new autoinstall for a completely unattended installation. But I would like for user-data YAML to be modified server-side based on MAC of a client

What I found

• I have boot kernel option of nocloud-net;s=http://... but I don't see a way to send a custom string as part of URL (or change URL altogether based on local MAC)
• I see early-commands which says that autoinstall will be refreshed after those are run, but I did not produce any valid way to use it to inject modified data to new autoinstall; eg. do wget http://myurl/$MAC then grep that file and modify autoinstall that's already running
• Using late-commands is option of last resort, where I could indeed do wget http://myurl/$MAC && .. && ... to eg. set static IP/GW/netmask, but it seems more error prone
• Edit: seems I could have another way, but also requires manual handling outside my expected web management, and that is to serve different pxelinux.cfg to each client and change the URL in there, but that's served through TFTP so no server-side scripting (unless there's a workaround?) Edit#2: This could work, point TFTP and HTTP servers to same folder(s), and tell PHP to generate custom files under /pxelinux.cfg/AA-BB-CC-DD-EE-01 ... -02 ... -03 .. etc for each MAC inside my database, and check/regenerate the files whenever an entry in DB is saved. Keeps idea of single management point at least. But I will leave question in case someone knows better solution (see above options)

Ultimate goal

I'd like to have a "master" PXE server, which is home to HTTPS server, and web management, where I could have a table of all devices (eg. in MySQL), and all settings related to each device. Then when we deploy new clients (mostly just some dumb kiosks and stuff), I'd pick their MACs, enable PXE boot, label them, and ship them to remote location. That MAC & location would be entered in MySQL via web management, together with stuff like static IP, GW, DNS, homepage of a browser, screen rotations, etc. Once they arrive and someone connects them, they would boot to unattended PXE install, go through it, in process they'd pull configs from web server (eg. user-data would actually be processed by PHP, and would inject required config as needed), and it would - well... just work. It could apply to live images as well, just in a different ways.

I am mostly stuck with the autoinstall and nocloud-net boot, it seems great at first, where I could serve a custom unattended file to each client, except for a fact I won't be able to recognize one client from the other. Fetching http://myserver/user-data?AA-BB-CC-DD-EE-FF doesn't seem to be in the specs, just picking it by IP wouldn't work as those would be random DHCP, unfortunately fixing that with DHCP reservations is a nightmare as we're talking about 100+ locations each with their own local DHCP, etc.

I'm running out of ideas, so hopefully someone can chime in. Any idea to get me going (appart from late-commands) would be great! It doesn't have to be MAC, can be UIID, or some other hardware ID (serial, etc), but it should be unique and easy to fetch. And MAC is usually a sticker on an outside of a box.

Oh, and if you wonder why I'm so against late-commands... well.. thing is I'm not against it, just that dynamically modifying unattended file would be so much more flexible. I could set hostname, IP, username, password, disk size, and all that right from the start. Way cleaner than booting with some defaults, then try to go through all places with bash scripts trying to fix it (specially the disks/partitions). After all, that's why we have autoinstall scripts in the first place, not to re-do it all after first boot.

Answer 1

So it seems I finally solved it. It's both easy and riddled with bugs and obstacles. Answer is yes - use early-commands. But truth is in the details, so detailed answer it is.

First of all, prepare the rest of your environment, you can see my other post about detailed steps I went through to do BIOS/UEFI PXE boot of 20.04 and 20.10: https://askubuntu.com/a/1292097/1080682

Now when you get your environment working correctly (good luck), let's do the custom autoinstall based on config changes served via HTTP mid install.

So if you follow the guide I posted on the link, I kept my Ubuntu user-data here :

/var/www/html/ubuntu-server-20.04.1/user-data

Modify the file with something like this (note I've shortened it for readability):

#cloud-config
autoinstall:
  version: 1
  refresh-installer:
    update: yes
  apt:
    <apt stuff>
  identity:
    hostname: pxe-client
    password: $6$zN/uHJD1rEXD/ETf$q8CoBt3xXmBT37RslyWcpLT1za4RJR3QEtosggRKN5aZAAf6/mYbFEQO66AIPm965glBXB1DGd0Sf.oKi.Rfx/
    realname: pxe
    username: pxe
  keyboard: {layout: hr, toggle: toggle, variant: ""}
  early-commands:
    - curl -G -o /autoinstall.yaml http://10.10.2.1/user-data -d "mac=$(ip a | grep ether | cut -d ' ' -f6)"
  locale: en_US
  network:
    network:
      version: 2
      ethernets:
        eth0:
          dhcp4: yes
          dhcp6: no

Now, this user-data can be real basic, all we need really is for it to have network enabled and that one-liner curl in early-commands. IP 10.10.2.1 is local IP of my HTTP server (also my PXE server, as I serve other config files and ISO images and all that through it, but doesn't matter).

Use whatever you want to modify and serve this file based on the request. Way it is being done above with curl you will actually request from server something like this:

GET /user-data?mac=fa:fa:fa:00:0e:07

The part fa:fa:fa:00:0e:07 is what server sends after querying it's own interfaces. If you have multiple interfaces you'll maybe need to tweak the script, or make sure only one interface is up during early install steps.

I'm planning to use that via PHP + MySQL, and after fetching it in PHP using $_GET["mac"] do something like SELECT * FROM autoinstall-configs WHERE mac = '$_GET["mac"]'; and from data in database table build the new autoinstall.yaml and serve it back to subiquity.

Anyway, your reply has to NOT HAVE the line autoinstall: !!

Here is minimal example of what HTTP/PHP will reply, I changed just hostname and username, and modified it so it passes subiquity syntax checks, oh and excluded early command not to get stuck in loop:

  version: 1
  refresh-installer:
    update: yes
  apt:
    <apt stuff>
  identity:
    hostname: php-client
    password: $6$zN/uHJD1rEXD/ETf$q8CoBt3xXmBT37RslyWcpLT1za4RJR3QEtosggRKN5aZAAf6/mYbFEQO66AIPm965glBXB1DGd0Sf.oKi.Rfx/
    realname: php
    username: php
  keyboard: {layout: hr, toggle: toggle, variant: ""}
  locale: en_US
  network:
    network:
      version: 2
      ethernets:
        eth0:
          dhcp4: yes
          dhcp6: no
  ssh:
    allow-pw: true
    install-server: true
  late-commands:
    - poweroff

To make it clear, here is the diff of the two files:

diff /var/www/html/ubuntu-server-20.04.1/user-data /var/www/html/user-data
1,3d0
< #cloud-config
<
< autoinstall:
16c13
<     hostname: pxe-client

>     hostname: php-client
18,19c15,16
<     realname: pxe
<     username: pxe

>     realname: php
>     username: php
21,22d17
<   early-commands:
<     - curl -G -o /autoinstall.yaml http://10.10.2.1/user-data -d "mac=$(ip a | grep ether | cut -d ' ' -f6)"

So it's almost the same, just not. These changes (removal of autoinstall: and early-commands: ) is required to pass on to rest of installation. You can test other tweaks on your own.

After that installation will continue with whatever new info was served via answer to request /user-data?mac=<installer-mac-address>.

This is now opening door to further possibilities of making your own web management for your VMs or server farms, or whatever. No longer do you need hand crafted user-data file for every server or group of servers. You can send them each their unique config, so including exact static IP address, partition sizes, different hostname, password, etc.

Canonical, if you pick an idea from this, put me in the credits at least :)

Case closed, cheers!

Answer 2

Depending on what you are using for PXE, you may be able to use variables in the boot arguments. For example, using GRUB (usually used for UEFI machines) would provide ${net_default_mac} as a variable for the MAC address.

Generically, I think early-commands is the best option. I'm not sure what failed when you tried it. I'd think you could just fetch your dynamically generated user-data file and overwrite the /autoinstall.yaml file.

Answer 3

I implemented a similar thing though I'm using Ansible/Ansible Tower as my source of truth. Essentially I have my iPXE script set to hit a simple flask app stored in OpenShift (kubernetes) with the MAC address of the machine requesting the autoinstall configuration. Ansible then runs and generates a machine-specific autoinstall (using a Jinja2 template) configuration file and serves it. No early scripts.

I do something similar with preseeds for Ubuntu releases older than 20.04.

import json
import requests
import subprocess
from flask import Flask, abort, request, redirect, send_from_directory
@autoinstaller.route('/ubuntu', defaults={'path': ''})
@autoinstaller.route('/ubuntu/<path:mac_address>')
@autoinstaller.route('/ubuntu/<path:mac_address>/meta-data')
@autoinstaller.route('/ubuntu/<path:mac_address>/user-data')
def get_autoinstall_config(mac_address):
    candidate_hostname = _isknownhost(mac_address)
    if candidate_hostname == None:
        abort(404)
    else:
        subprocess.run(["ansible-playbook", "playbooks/ubuntu/generate_autoinstall.yml", "--limit", f"{candidate_hostname}", "-e", f"candidate_mac={mac_address} candidate_hostname={candidate_hostname} authorized=True ansible_connection=local"])
        return send_from_directory(f'/tmp/{mac_address}', 'user-data')

_isknownhost is a function that calls ansible-inventory to find a system with a matching mac address.

def _isknownhost(mac_address):
    query = subprocess.Popen(["ansible-inventory", "--list"], stdout=subprocess.PIPE)
    hosts = json.loads(query.communicate()[0])['_meta']['hostvars']
    for system in hosts.keys():
        for hostvar in hosts[system].values():
            if hostvar == mac_address:
                return system
    return None