TrueNAS on Proxmox with HBA (LSI/Broadcom 9300-16i)

This guide covers:

• Checking HBA firmware & mode
• Updating firmware (IT mode)
• Enabling PCI passthrough in Proxmox
• Migrating HDDs from motherboard SATA to HBA
• Setting up VFIO bindings
• Verifying PCIe lanes per chip
• Listing disks per HBA chip

---

0. Download Tools & Firmware

Resources

• Broadcom 9300-16i Downloads
• LSI 9300-xx Firmware Update | TrueNAS Community
• Secure Boot support for UEFI BSD mpt3x64.rom · GitHub
• edk2/ShellBinPkg/UefiShell/X64/Shell.efi at UDK2018 · tianocore/edk2

Downloaded files

• bootx64.efi – UEFI shell boot file
• SAS9300_16i_IT.bin – IT-mode firmware
• mptsas3.rom – Legacy BIOS ROM
• mpt3x64.rom – UEFI BIOS ROM
• sas3flash/sas3flash.efi – Firmware flashing tool
• sas3ircu – Integrated RAID Configuration Utility

---

1. Check HBA Firmware & Mode

After installing the 9300-16i, confirm both chips are detected:

lspci | grep -i sas

Example (dual-chip):

19:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)
1b:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 (rev 02)

Check firmware & mode:

./sas3flash -listall
./sas3flash -c 0 -list
./sas3flash -c 1 -list

Sample output:

Adapter Selected is a LSI SAS: SAS3008
Firmware Product ID: 0x2221 (IT)
Firmware Version: 16.00.12.00

✅ IT mode required (not IR)
✅ Firmware version e.g. 16.00.12.00

---

2. Update Firmware (IT Mode)

Erase existing firmware (only if converting IR → IT)

./sas3flash -c 0 -o -e 6
./sas3flash -c 1 -o -e 6

• -e 6 → clean flash (preserves manufacturing data)
• -e 7 → full erase (use with caution)

Flash IT firmware

Without BIOS ROM (recommended for TrueNAS/Proxmox):

./sas3flash -c 0 -o -f SAS9300_16i_IT.bin
./sas3flash -c 1 -o -f SAS9300_16i_IT.bin

With BIOS ROM (optional):

./sas3flash -c 0 -o -f SAS9300_16i_IT.bin -b mptsas3.rom -b mpt3x64.rom
./sas3flash -c 1 -o -f SAS9300_16i_IT.bin -b mptsas3.rom -b mpt3x64.rom

Flags explained:

• -c <num> → target controller (0 or 1)
• -o → perform operation
• -f <file> → firmware file
• -b <file> → BIOS ROM file

Verify:

./sas3flash -listall

Confirm both controllers show IT mode and the latest firmware.

---

Flashing via UEFI Shell (Optional)

1. Format a USB stick as FAT32.
2. Place bootx64.efi on it under /efi/boot/.
3. Disable Secure Boot temporarily.
4. Boot to the UEFI Shell.
5. Identify and switch to the USB drive:

   fs0: 
   dir

   You should see your .efi and .rom files.
6. Run the same sas3flash.efi commands as above.

Reference:

• EverLand1/9300-8i_IT-Mode (GitHub)

---

3. Enable PCI Passthrough (IOMMU)

Follow Proxmox-passthrough-PCI.

---

4. VFIO Binding (Auto-Passthrough at Boot)

This step is optional: Proxmox VE 7+ typically binds PCI devices to vfio-pci automatically when they are assigned to a VM.

Identify PCI devices

lspci -nn | grep -i sas

Example:

19:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)
1b:00.0 Serial Attached SCSI controller [0107]: Broadcom / LSI SAS3008 PCI-Express Fusion-MPT SAS-3 [1000:0097] (rev 02)

🔹 Quick verification:

lspci -k | grep -A3 -E "19:00.0|1b:00.0"

If it shows Kernel driver in use: vfio-pci, the passthrough is ready.

Bind to vfio-pci

Modify /etc/modprobe.d/vfio.conf, add:

options vfio-pci ids=1000:0097 disable_vga=1

Blacklist native drivers

Modify or create /etc/modprobe.d/blacklist.conf, add:

blacklist mpt3sas
blacklist ahci

Rebuild initramfs and reboot

update-initramfs -u -k all reboot

Verify binding

lspci -k | grep -A3 -E "19:00.0|1b:00.0"

Expected:

Kernel driver in use: vfio-pci

---

5. Migrate HDDs from SATA to HBA

Hardware Steps

1. Disable Start at boot for the TrueNAS VM.
2. Power off the host.
3. Move SATA cables from motherboard ports to HBA (Mini-SAS → SATA).
4. Power on the host.

Proxmox Steps

1. Verify HBA is detected:

   lspci | grep -i sas

2. In the Proxmox GUI:
   • VM → Hardware → Add → PCI Device
   • Select Raw device
   • Select the desired SAS controller(s)
   • Options:
     • ✅ All Functions
     • ⬜️ ROM-Bar Off (only if passthrough issues)
     • ✅ PCIe (for Q35 machine type)
3. Remove any old SATA passthrough entries.

TrueNAS Steps

1. Boot the VM — drives should appear under the HBA.
2. Go to Storage → Pools → Import if pools are not detected automatically.
3. Verify datasets, shares, and SMART.

---

6. Verify PCIe Link Width

lspci -nn | grep -i sas
# Find your 9300-16i chips (usually 19:00.0 and 1b:00.0)

# Check current negotiated link width/speed per chip
lspci -s 19:00.0 -vv | grep -i "LnkSta:"
lspci -s 1b:00.0 -vv | grep -i "LnkSta:"

# Optional: check maximum capability
lspci -s 19:00.0 -vv | grep -i "LnkCap:"
lspci -s 1b:00.0 -vv | grep -i "LnkCap:"

• Negotiated Width = lanes currently active (x4/x8)
• Capable Width = maximum the slot/controller supports (x8/x16)

---

7. List Disks per HBA Chip (sas3ircu)

sas3ircu list
sas3ircu 0 display   # Chip 0
sas3ircu 1 display   # Chip 1

---

8. Notes & Recommendations

• No need to export/import ZFS pools when reusing the same TrueNAS VM.
• Back up TrueNAS config: System → General → Save Config.
• 9300-16i runs hot — maintain good airflow; keep temps < 70 °C.
• Lane requirements:
  • PCIe 3.0 ×8 = full bandwidth
  • Works in ×4 slots but reduced throughput
  • Fine for HDDs; use ×8 for SSD arrays