Install Debian on a physical host machine

Server Installation

Initial steps

  • Add the server to the hosts.yml file and create a host_vars file. Ensure there is a console entry and then run ansible to the console server (in Telehouse it’s clr). Read the top of the IMPI page as well to get the settings right.
  • Ensure that network boot is setup in the cabinet in which you are installing the server.
  • Plug in monitor and keyboard
  • Enter Bios/Setup. Specify that Bios should output to serial console.
  • Ensure that server skips errors like no keyboard attached
  • If it’s IMPI, configure bios to use IPMI.
  • Reboot and select Boot Options and choose to boot to network from F12 menu
  • NOTE: If it’s IPMI (most new servers), you won’t be able to access the debirf image via the SOL interface unless you fix the console settings - by default, the debirf image will try to boot with console=ttyS0, but IPMI will be listening on ttyS1, so you have to adjust it:
    • Connect via iKVM via the web first
    • At the grub prompt, edit the debirf line.
    • Change to: console=ttyS1,115200n8 (change ttyS0 to ttyS1)

Drive partioning/Disk setup

The big picture is…

  • All partitions should be Primary

  • One tiny partition used for bios/grub data so we can boot from them.

  • A second approximately 1024MB partition: configured as RAID1 across all disks, used as boot partition

  • The rest of the space on each disk: configured as RAID1 (or RAID10 if using four identical disks), used as encrypted disk

  • Encrypted disk: used as physical volume for LVM

  • If…

    • Two SATA and two SSD: Create volume group for the SATA disks (vg0) another one for the SSD disks (vg1)
    • Four SSD: Create volume group for the SSD disks (vg0)

  • Create standard partitions as logical volumes

Details - Install manually via debootstrap

Networking

Due to our firewall setup, you will need to add a network route to reach our caching DNS servers. Replace “DEV” with the name of the active network device (use ip link to see a list of possible options).

ip route add 204.19.241.0/24 dev DEV

Debian Repos and packages

Add update/security repos to the debian live repo, and remove the CD repo (note debian live repo is still running bookworm, even though we will be installing trixie):

echo "deb http://deb.debian.org/debian bookworm main
deb http://security.debian.org/debian-security bookworm-security main
deb http://http.us.debian.org/debian bookworm-updates main" > /etc/apt/sources.list

Next install needed packages:

apt update && apt install cryptsetup parted dosfstools debootstrap

Partitioning

Check disk’s sector size! If it’s 512B per sector, these are good numbers. If it’s 4096 they chould be changed.

Bios servers

Create a 4MB biosboot partition and a 1GB boot partition.

for disk in a b c d; do
  parted /dev/sd${disk} -- mklabel gpt
  parted /dev/sd${disk} -- unit s mkpart biosboot 8192 16383
  parted /dev/sd${disk} -- set 1 bios_grub on
  parted /dev/sd${disk} -- unit s mkpart boot 16384 2113535
  parted /dev/sd${disk} -- set 2 raid on
  parted /dev/sd${disk} -- unit s mkpart pv 2113536 -196608
  parted /dev/sd${disk} -- set 3 raid on
done
UEFI servers

Create 1 GB EFI partition and a 1 GB boot partition.

for disk in a b c d; do
  parted /dev/sd${disk} -- mklabel gpt
  parted /dev/sd${disk} -- unit s mkpart efi 8192 2105343
  parted /dev/sd${disk} -- unit s mkpart boot 2105344 4202495
  parted /dev/sd${disk} -- set 2 raid on
  parted /dev/sd${disk} -- unit s mkpart pv 4202496 -196608
  parted /dev/sd${disk} -- set 3 raid on
done

RAID

Two SATA, Two SSD:
mdadm --create --raid-devices=4 --level=1 --metadata=1.0 --verbose /dev/md0 /dev/sd[abcd]2
mdadm --create --raid-devices=2 --level=1 --metadata=1.0 --verbose /dev/md1 /dev/sd[ab]3
mdadm --create --raid-devices=2 --level=1 --metadata=1.0 --verbose /dev/md2 /dev/sd[cd]3
Four SSD:
mdadm --create --raid-devices=4 --level=1 --metadata=1.0 --verbose /dev/md0 /dev/sd[abcd]2
mdadm --create --raid-devices=4 --level=10 --metadata=1.0 --verbose /dev/md1 /dev/sd[abcd]3

Cryptsetup

cryptsetup luksFormat /dev/md1
cryptsetup luksOpen /dev/md1 md1_crypt

If md2 was created…

cryptsetup luksFormat /dev/md2
cryptsetup luksOpen /dev/md2 md2_crypt

lvm

pvcreate /dev/mapper/md1_crypt
vgcreate vg0 /dev/mapper/md1_crypt
lvcreate --name swap --size 1GB vg0
lvcreate --name root --size 20GB vg0
lvcreate --name var --size 10GB vg0
lvcreate --name tmp --size 1GB vg0

Filesystems

Then, create fileystems:

  • For the boot partition mkfs -t ext4 /dev/md0
  • Repeat for each logical volume
for part in var tmp root; do
  mkfs -t ext4 /dev/mapper/vg0-${part}
done
mkswap /dev/mapper/vg0-swap
mkfs -t ext4 /dev/md0
If efi

Create a vfat partition on all the efi partitions.

for disk in a b c d; do
  mkfs -t vfat /dev/sd${disk}1
done

Install and run debootstrap

Before you can install Debian, mount all partitions in /mnt

mount /dev/mapper/vg0-root /mnt
mkdir -p /mnt/{boot,var,proc,dev,sys,tmp,run/udev}
mount /dev/md0 /mnt/boot
mount /dev/mapper/vg0-var /mnt/var
mount /dev/mapper/vg0-tmp /mnt/tmp
If using efi…
mkdir /mnt/boot/efi
# Note, UFI does not *seem* to support RAID 1.
# https://askubuntu.com/questions/66637/can-the-efi-system-partition-be-raided
mount /dev/sda1 /mnt/boot/efi

Note: Don’t mount /proc, /sys, and /dev before running debootstrap - you will end up with a broken installation (apt won’t be available for one thing).

Run debootstrap: debootstrap trixie /mnt

Chroot

Bind mount kernel directories:

mount -o bind /proc /mnt/proc
mount -o bind /sys /mnt/sys
mount -o bind /dev /mnt/dev
# See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=918590
mount -o bind /run/udev /mnt/run/udev

Enter a chroot environment: chroot /mnt

Prep work

Set proper permissions on /tmp: chmod 1777 /tmp

Set some configuration files that should be in place before packages are installed.

  • /etc/fstab
printf "UUID=$(blkid --output value /dev/mapper/vg0-root | head -n1)  / ext4  errors=remount-ro 0 1
UUID=$(blkid --output value /dev/md0 | head -n1)  /boot ext4  defaults  0 2
UUID=$(blkid --output value /dev/mapper/vg0-var | head -n1) /var  ext4  defaults  0 2
UUID=$(blkid --output value /dev/mapper/vg0-tmp | head -n1) /tmp  ext4  defaults  0 2
/dev/mapper/vg0-swap  none  swap  sw  0 0
" > /etc/fstab
  • If using efi…
printf "/dev/sda1  /boot/efi vfat defaults 0 2\n" >> /etc/fstab
  • /etc/crypttab
printf "# <target name> <source device> <key file>  <options>\n" > /etc/crypttab
for md in $(ls /dev/mapper/*_crypt| egrep -o 'md[0-9]+'); do printf "${md}_crypt UUID=$(blkid --output value /dev/${md}|head -n1)  none  luks\n" >> /etc/crypttab; done

Install packages

Ensure we are pulling in updates - check /etc/apt/sources.list and add:

deb http://security.debian.org/debian-security trixie-security main
deb http://http.us.debian.org/debian trixie-updates main

Install a lot of necessary packages for booting:

apt update

If using bios…

apt install mdadm lvm2 cryptsetup cryptsetup-initramfs grub-pc linux-image-amd64 bridge-utils openssh-server screen rsync vim

If using efi…

apt install mdadm lvm2 cryptsetup cryptsetup-initramfs grub-efi-amd64 linux-image-amd64 bridge-utils openssh-server screen rsync vim

Update files required for booting

  • /etc/systemd/network/br0.netdev
[NetDev]
Name=br0
Kind=bridge
  • /etc/systemd/network/br0.network
[Match]
Name=br0

[Network]
Address=xx.xx.xx.xx/xx
Gateway=xx.xx.xx.x
  • /etc/systemd/network/cable0.network
[Match]
Name=eno1

[Network]
Bridge=br0
  • /etc/default/grub.d/local.cfg
# Add the following (ttyS1 instead if using IPMI)
# and ensure "QUIET" is not included anywhere in the file.
GRUB_CMDLINE_LINUX="console=ttyS0,115200n8"
# And
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1"
  • /etc/hosts
127.0.0.1	localhost
xx.xx.xx.xx	yyy.mayfirst.org yyy 
::1		localhost ip6-localhost ip6-loopback
ff02::1		ip6-allnodes
ff02::2		ip6-allrouters
  • /etc/hostname
justnamenotdomain

Almost done!

  • Enable networking: systemctl enable systemd-networkd
  • Generate /etc/mdadm/mdadm.conf file. Generate with: /usr/share/mdadm/mkconf > /etc/mdadm/mdadm.conf
  • Set root passwd: passwd
  • Add your ssh public key to /root/.ssh/authorized_keys
  • Note the ssh fingerprints: for key in $(ls /etc/ssh/ssh_host_*.pub); do ssh-keygen -l -f "$key"; done
  • Create a DNS record from control panel.
  • Install grub: grub-install /dev/sda; grub-install /dev/sdb; grub-install /dev/sdc; grub-install /dev/sdd;
  • Ensure latest changes are reflected: update-grub && update-initramfs -u
  • Reboot!

After reboot

Copy the file roles/kvm/files/mf-initialize-server to the server and run it. It will install some basic packages and then ask for your control panel login. With the login, it will create DNS ssh fingerprint records for the server.