Switch

Setup ONIE

1. Flash ONIE iso to USB (DL from here DL from https://misc.bigdata.sh/as7712-32x-r0_onie_v2020.08.00.02.iso.zip)
2. Connect to console: screen /dev/cu.usbserial-* 115200
3. Boot switch to USB (Del to get into BIOS to set boot priority) -> Rescue
4. Wipe the drive: parted /dev/sda mklabel
5. Reboot to USB again -> Embed ONIE

Install SONiC

1. Get link for most recent SONiC 202311 broadcom: https://sonic.software/
2. Connect switch to internet (Mgmt ethernet port)
3. onie-nos-install http://link-for/sonic-broadcom.bin
4. Reboot, let SONiC install, change passwd

To make it into a dumb switch:

• sudo config vlan add 100
• sudo config interface fec Ethernet0 rs && ...
• sudo config interface ip remove Ethernet0 10.0.0.0/31 && ...
• sudo config vlan member add -u 100 Ethernet0 && ...

Or use this to generate all of the commands:

for i in range(0,125,4):
	print(f"sudo config interface fec Ethernet{i} rs && \\")
	print(f"sudo config interface ip remove Ethernet{i} 10.0.0.{i/2}/31 && \\")
	print(f"sudo config vlan member add -u 100 Ethernet{i} && \\")

Misc

• show interfaces transceiver presence should show present for ports with optics
  • show interfaces transceiver eeprom Ethernet0 --dom
  • Might take a minute to show up?

On machines

1. wget https://content.mellanox.com/ofed/MLNX_EN-5.8-4.1.5.0/mlnx-en-5.8-4.1.5.0-ubuntu22.04-x86_64.tgz && tar xf mlnx-en-5.8-4.1.5.0-ubuntu22.04-x86_64.tgz && cd mlnx-en-5.8-4.1.5.0-ubuntu22.04-x86_64 && sudo ./install
2. lspci | grep Mellanox to get ID and mstconfig -d PCIe_ID_THING set LINK_TYPE_P1=2
3. Reboot.

See https://fasterdata.es.net/host-tuning/linux/100g-tuning/ for TCP/IP tuning.

Debug

• sudo /usr/bin/mstconfig -d PCIe_ID_THING query
• sudo ethtool -m PCIe_ID_THING

Useful links:

https://github.com/sonic-net/sonic-utilities/blob/202311/doc/Command-Reference.md https://github.com/sonic-net/SONiC/wiki/Troubleshooting-Guide