A better way to do this – Update: 2021-06-28
Someone has put together a patched kernel for Proxmox to let people run passthrough on their HP servers, it’s available on GitHub as a .deb download for direct install or you can build it from their sources yourself. Visit kiler129’s relax-intel-rmrr repository for a better way to do this 🙂
This github was valid for Proxmox 6, but if you want a working kernel for Proxmox 7, I use the builds available from Aterfax’s github, as they’re up to date and I’m using them myself 🙂
The rest of this post is included as an archive.
This is a quick stub post to show how I finally got PCI pass-through working on my HP DL360 G6 server, since it would show an error such as
DMAR-IR: This system BIOS has enabled interrupt remapping interrupt remapping is being disabled.
If you actually tried to launch a machine with a PCI device attached, it would say:
Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.
From what I can tell this is because the system uses RMRR to control fan speeds, and having these bits set causes the driver to disable IOMMU. There is a simple change you can make to the Proxmox kernel source code to get passed this error and use PCI devices with VMs without issue.
sed -i.new 's/device_is_rmrr_locked(dev)/false/' ./$UBUNTU_NAME/drivers/iommu/intel-iommu.c
After making that change and recompiling the kernel, it works perfectly fine, but the system will still say DMAR-IR interrupts have been disabled, however it will work fine.
I am indebted to this Reddit post and the script that they put together that was created from their research, which let me get my DL360 G6 working. They have a DL380 but I am pleased it works on my system. I don’t have a Reddit account, but show the original author some love if you do! https://old.reddit.com/r/homelab/comments/iw5cew/proxmox_i_created_a_script_to_fix_gpu_pass/
I’ve quoted their script below just in-case the original pastebin link in their Reddit post goes down.
#!/bin/bash
#Proxmox 6.2 over Ubuntu Focal Fossa patch for IOMMU.
#this can be done with Proxmox vanilla, just run the script!
#based heavily on
#https://forum.proxmox.com/threads/help-with-pci-passthrough.23980/
#move to working dir
cd "${0%/*}"
#Proxmox 4.4 kernel modding guide dependencies (see the above link)
#apt-get install git screen fakeroot build-essential devscripts libncurses5 libncurses5-dev libssl-dev bc flex bison libelf-dev libaudit-dev libgtk2.0-dev libperl-dev libperl-dev asciidoc xmlto gnupg gnupg2
#Debian kernel build dependencies
#apt-get install build-essential linux-source bc kmod cpio flex cpio libncurses5-dev
#zfsonlinux has hidden dependencies
#apt-get install dh-python python3-cffi python3-setuptools python3-sphinx python3-all-dev
#even more dependencies
#apt-get install libdw-dev libiberty-dev libnuma-dev libslang2-dev lz4
#full list of dependencies, though it may contain duplicates
apt-get install git screen fakeroot build-essential devscripts libncurses5 libncurses5-dev libssl-dev bc flex bison libelf-dev libaudit-dev libgtk2.0-dev libperl-dev libperl-dev asciidoc xmlto gnupg gnupg2 build-essential linux-source bc kmod cpio flex cpio libncurses5-dev dh-python python3-cffi python3-setuptools python3-sphinx python3-all-dev libdw-dev libiberty-dev libnuma-dev libslang2-dev lz4
#rm -r ./tmp/proliant-iommu-patch/
#TODO Only if the user says so on finishing compilation - Ubuntu-focal file is weird so we have to remove
#rm -r ./pve-kernel
#mkdir ./tmp/proliant-iommu-patch
#cd ./tmp/proliant-iommu-patch
#/tmp/proliant-iommu-patch/pve-kernel/patches/kernel/
git clone git://git.proxmox.com/git/pve-kernel.git
#mkdir ./pve-kernel
cd ./pve-kernel
#use a proxmox makefile bug to only get the submodules. THIS iS A BAD IDEA
make -j 2
cd ./submodules
#saves the hassle of Proxmox using git on its own
#git clone git://kernel.ubuntu.com/ubuntu/ubuntu-focal.git/
#TODO UBUNTU_NAME= "$(ls | grep -m 1 ubuntu)" > used for compatibility
UBUNTU_NAME="$(ls | grep -m 1 ubuntu)"
echo $UBUNTU_NAME
#if [ ! -d ./$UBUNTU_NAME/drivers ]
#then
# #git clone git://kernel.ubuntu.com/ubuntu/ubuntu-focal.git
# #git clone git://git.proxmox.com/git/mirror_$UBUNTU_NAME-kernel
# git clone https://github.com/torvalds/linux.git
# echo ""
# #git clone $REPOSRC $LOCALREPO
#else
# echo $UBUNTU_NAME + " is already cloned, skipping!"
# #cd $LOCALREPO
# #git pull $REPOSRC
#fi
#
#mv -T ./mirror_$UBUNTU_NAME-kernel ./$UBUNTU_NAME
sed -i.new 's/device_is_rmrr_locked(dev)/false/' ./$UBUNTU_NAME/drivers/iommu/intel-iommu.c
#cat ./iommu/intel-iommu.c | grep 'device_is_rmrr_locked'
#named according to proxmox tutorial :)
#diff -u ./iommu/intel-iommu.c ./ubuntu-focal/drivers/iommu/intel-iommu.c > ./remove_mbrr_check.patch
#move back to working dir - TODO this
#cd "${0%/*}"
cd ../
echo "building"
make
find ./ -name '*.deb' -exec cp -prv '{}' '../' ';'
#optional TODO
##dpkg -i *.deb
#update-grub
cd ../
echo "
#find ./ -name '*.deb' -exec dpkg -i '{}' ';'
dpkg -i *.deb
update-grub
echo \"=======================================================================\"
echo \"Please reboot your system. Tell u/oezingle if your machine doesn't boot\"
echo \"=======================================================================\"
" > ./install-new-kernel.sh
chmod 0777 ./install-new-kernel.sh
rm -R ./pve-kernel
echo ""
echo ""
echo "===================================================================="
echo "Install all .deb files and update-grub, or run install-new-kernel.sh"
echo "===================================================================="