█████ ██████   █████ ███████████ ███████████     █████████   ██████████   ██████████

░░███ ░░██████ ░░███ ░█░░░███░░░█░░███░░░░░███ ███░░░░░███ ░░███░░░░███ ░░███░░░░░█ ░███ ░███░███ ░███ ░ ░███ ░ ░███ ░███ ░███ ░███ ░███ ░░███ ░███ █ ░ ░███ ░███░░███░███ ░███ ░██████████ ░███████████ ░███ ░███ ░██████
░███ ░███ ░░██████ ░███ ░███░░░░░███ ░███░░░░░███ ░███ ░███ ░███░░█
░███ ░███ ░░█████ ░███ ░███ ░███ ░███ ░███ ░███ ███ ░███ ░ █ █████ █████ ░░█████ █████ █████ █████ █████ █████ ██████████ ██████████ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░ ░░░░░░░░░░ ░░░░░░░░░░

Purpose

Flash Mellanox ConnectX-3 cards to latest and full firmware, these cards can be configured as 40GbE card instead of 56Gb InfiniBand cards.

Notes Ubuntu 20.04 LTS was used for flashing, 24.04 LTS kernel and the mft tools are not compatible After flashing the card works on newer kernels, it's only the flashing process

Configuration

Download latest Mellanox Firmware tools

sudo apt update && sudo apt install gcc make dkms unzip linux-headers-$(uname -r)
wget https://www.mellanox.com/downloads/MFT/mft-4.18.0-106-x86_64-deb.tgz
tar -xvf mft-4.18.0-106-x86_64-deb.tgz && cd mft-4.18.0-106-x86_64-deb
sudo ./install.sh

Start MST service:

mst start
mst status

Download latest FCBT firmware and unzip:

wget http://www.mellanox.com/downloads/firmware/fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip
unzip fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin.zip

Backup the cards current board definition file just in case

flint -d /dev/mst/mt4099_pci_cr0 dc orig_firmware.ini

Flash the FCBT image to the card

flint -d /dev/mst/mt4099_pci_cr0 -i fw-ConnectX3-rel-2_42_5000-MCX354A-FCB_A2-A5-FlexBoot-3.4.752.bin -allow_psid_change burn

Reboot

start mst service again

mst start

Get detailed firmware info

mlxfwmanager --query

Get the device ID again:

mst status

Use it to see the cards current configuration:

mlxconfig -d /dev/mst/mt4099_pci_cr0 query

Switch ports from VPI/Auto to Ethernet only:

mlxconfig -d /dev/mst/mt4099_pci_cr0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2

Turn off bootrom crap

mlxconfig -d /dev/mst/mt4099_pci_cr0 set BOOT_OPTION_ROM_EN_P1=false
mlxconfig -d /dev/mst/mt4099_pci_cr0 set BOOT_OPTION_ROM_EN_P2=false
mlxconfig -d /dev/mst/mt4099_pci_cr0 set LEGACY_BOOT_PROTOCOL_P1=0
mlxconfig -d /dev/mst/mt4099_pci_cr0 set LEGACY_BOOT_PROTOCOL_P2=0

Optional: delete bootrom off the card, so it doesn't slow down boot by popping up crap (safe & supported)

flint -d /dev/mst/mt4099_pci_cr0 --allow_rom_change drom