-
-
Save FlorianHeigl/ca7fe009077437983a3dc927d12f339b to your computer and use it in GitHub Desktop.
Es gibt zwei Userlevel
- das ist das Bootpasswort
- wenn es gesetzt ist, startet sas System nur mit dem PW)
- das ist das BIOS Passwort
- Wird beim Aufruf des BIOS abgefragt
- Wenn man nur "enter" drueckt, kommt man ins BIOS, hat aber nicht den "admin" Userlevel
- Das PW sollte normalerweise
SERVICETAG!sein - auf dem Auszieher mit den QR Codes oben rechts an der Gehaeusefront
- das Service Tag findet man auch auf einem Sticker auf dem Deckel
DiagOS selbst kann mittels einem .deb Package geupdated werden
FW Updates
updatetool --dev=BIOS --update -e filename.img
updatetool --dev=CPLD --update -e filename.jed
updatetool --dev=BMC --update -e filename.bmc
Wenn man am BMC Passwort scheitert (serieller Port) ist das BMC Update ein recht guter Loesungspfad
Das DiagOS zieht sich keine neue IP vom DHCP Server, wenn man den Lan-Port aus- und einsteckt.
- Updates entpacken / Files identifizieren (s.u.)
- Per TTY in DiagOS einloggen, aktiven LAN-Port identifizieren, IP auslesen.
- Neue IP bei Bedarf via ifdown / ifup holen
- Die Update Files per scp in das gebootete DiagOS kopieren
root@dellemc-diag-os:~# updatetool --dev=BMC --update -e VEP4600-BMC-v2.30.ima
Disable device protect
Write image to BMC
INFO: Yafu INI Configuration File not found... Default options will not be applied...
Creating IPMI session via USB...Done
-------------------------------------------------
YAFUFlash - Firmware Upgrade Utility (Version 5.0.0)
-------------------------------------------------
(C)Copyright 2016, American Megatrends Inc.
Image To be updated is (Image-1)
The Module boot major or minor version is different from the one in the image
So,Type (Y/y) to do Full Firmware Upgrade or (N/n) to exit
Enter your Option : y
****************************************************************************
WARNING!
FIRMWARE UPGRADE MUST NOT BE INTERRUPTED ONCE IT IS STARTED.
PLEASE DO NOT USE THIS FLASH TOOL FROM THE REDIRECTION CONSOLE.
****************************************************************************
Preserving Env Variables... done
Uploading Firmware Image : 100%... done
Flashing [boot] Module ....
Flashing Firmware Image : 100%... done
Verifying Firmware Image : 100%... done
Flashing [conf] Module ....
Flashing Firmware Image : 100%... done
Verifying Firmware Image : 100%... done
Flashing [root] Module ....
...
Flashing [ast2500e] Module ....
Flashing Firmware Image : 100%... done
Verifying Firmware Image : 100%... done
Setting Env variables... done
Resetting the firmware..........
write BMC image success
Enable device protect
Update BMC image success
Beim BMC Update geht die Lueftersteuerung kurz aus (Full)
Das hier hat wohl nicht funktioniert
root@dellemc-diag-os:~# updatetool --dev=CPLD --update -e AZUL_CPLD_V12_20210520.jed
-------------------------------------------------
YAFUFlash - Firmware Upgrade Utility (Version 5.0.0)
-------------------------------------------------
(C)Copyright 2016, American Megatrends Inc.
Beginning CPLD Update...
Uploading Image : 100%... done
Flashing Firmware Image : 0%Error in ECFStatus
Error in updating CPLD image
write CPLD image success
Enable device protect
Update CPLD image success
BIOS Update fragt nicht nach Yes/No
root@dellemc-diag-os:~# updatetool --dev=BIOS --update -e VEP4600-BIOS-3.41.0.9-23.BIN
Write image to BIOS
+---------------------------------------------------------------------------+
| AMI Firmware Update Utility Manufacture for Dell Nv5.11.06.1854|
| Copyright (C)2018 American Megatrends Inc. All Rights Reserved. |
+---------------------------------------------------------------------------+
Reading flash ............... done
- ME Data Size checking . ok
- FFS checksums ......... ok
- Check RomLayout ........ ok.
Erasing Main Block .......... done
Updating Main Block ......... 0x0028C000 (13%)
...
- Update success for MER
WARNING : System must power-off to have the changes take effect!
write BIOS image success
Update BIOS image success
Am Ende also das System mit poweroff runterfahren - oder irgendwann spaeter.
Im Fall mit dem CPLD Update Fehler am besten ganz stromlos machen.
(Funktionierte so, aber hab es nicht ohne stromlos machen versucht.)
Uebersicht / Beschreibung:
https://infohub.delltechnologies.com/p/dell-technologies-vep4600-bios-firmware-upgrade/
DiagOS und Unified FW Updater
DiagOS Partitioning issues
Echte Loesung war: DiagOS mit Rufus auf frischen USB Stick. Diesen in UEFI Modus booten.
Release Notes aktuelle Version 3.8
Release Notes VEP 4600 DiagOS
Download Link Treiber und Tools, UFW
System fuer Arbeiten:
- Ubuntu 20: Geht
- CentOS Stream: Geht nicht!
Software arbeitet normal in einer RAM Disk, also genug Ram fuer VM zuordnen.
Alle Zipfiles auspacken
das ufw_v.x ist das binary, aufrufen mit
./ufwding 192.168.86.xx interactive
NOTE If an IPMI session error is seen while connecting to the BMC, run following command to reset the BMC prior to re-run of UFW.
Creating IPMI session via network with address 192.168.86.xx...Failed
root@ubs18-04:~/# ipmitool -I lanplus -H xxx.xx.xx.xxx -U admin -P admin mc reset cold
need reset!!!!!
[...]
9. Reset BMC after CPLD update completion and system bootup
q. Exit
Enter your choice:9
Reset BMC, in case of BMC is not in correct status (e.g. BMC in Firmware Update Mode will cause firmware update that through CFU failed)
Sending reset to BMC to recover, need to wait for 180 seconds
Reset sieht man auf der BMC Konsole auch:
[ 9899.290059] ipmi_si IPI0001:00: Maybe ok, but ipmi might run very slowly.
Reset per Menu hilft NICHT
updating BMC image
****************************
* BMC version = 2.0 *
* Back-up Version = 2.0 *
****************************
updating primary image only..
INFO: Yafu INI Configuration File not found... Default options will not be applied...
Creating IPMI session via network with address 192.168.xx.100...Failed
Image update failed
Upgrade Failed & Exiting
/home/floh/VEP4600/temp
Kabel einstecken sucht scheinbar nach IP Plugging a cable triggers DHCP refresh.
~ # DHCP monitor: Re[ 366.450000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
newing eth0 interface for IPv6
RTNETLINK answers: File exists
Sending ICMPv6 packet: Cannot assign requested address
Renewing DNS for eth0 interface.
DHCP monitor: Renewing eth0 interface for IPv6
RTNETLINK answers: File exists
Sending ICMPv6 packet: Cannot assign requested address
Renewing DNS for eth0 interface.
kein funktionierenden DHCP beobachtet
From the BMC console window, run the following commands to boot into the BIOS:
ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin chassis bootparam set bootflag force_bios
~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin power reset
Chassis Power Control: Reset
~ # RESET CHASSIS
[ 653.800000] LPC RESET
PDK LPC Reset is invoked
Seriell muss man dann bestimmt umstecken
Note: ipmi user/pw != bmc
Laut Handbuch muss man das alles per BIOS machen.
Geht aber auch mit ipmitool lokal
ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 ipaddr 10.0.0.1
41~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 netmask 255.255
.255.0
~ # ipmitool -I lanplus -H 127.0.0.1 -U admin -P admin lan set 1 defgw ipaddr 10.0.0.1
Port names are in clock-wise order
| Port | Description |
|---|---|
| Port A | Serial Console Port to Access CPU |
| Port B | Serial Console Port to Access BMC |
| Port C | Management Port to Access CPU (igb4) |
| Port D | Management Port to Access BMC |
| Port E | USB-B Console port to Access CPU |
If a cable is connected to Port E (referenced above) even if not in use, the CPU RJ45 (Port A) console port will not work.
Port C is an Intel i210 desktop NIC with reduced features intended for management purposes only. The other 1g Ports use i350 server NICs. Rarely will the i210 port be enumerated as the first NIC
Accessing the console ports:
Port A/E - Serial/USB-B Console Port to access CPU
User Name : root
Password : calvin
The above login credentials are only for the default installed DIAG OS When installed with any other OS, follows vendor directions.
Port B - Serial Console Port to access BMC
User Name : sysadmin
Password : superuser
In bestimmten Versionen ist der BMC Login anders:
User Name : sysadmin
Password : SERVICETAG!
Accessing BMC through HTTPS
User Name : admin
Password : admin
Default Console Settings
Baud Rate : 115200
Data Bits : 8
Stop Bits : 1
Parity : None
Flow Control : None
BMCXXXX login: sysadmin
Password:
[1446 : 1446 INFO]SERIAL Login from IP:127.0.0.1 user:sysadmin
IP Setup?
3: eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
link/ether XXXX brd ff:ff:ff:ff:ff:ff
inet 192.168.0.1/24 brd 192.168.0.255 scope global eth0
- Power LED
- Primary unit indicator LED
- System LED
- Locator LED
- Temperature LED
- Fan LED
- SFP+ indicator LED
- 10/100/1000 BaseT RJ-45 networking link (left) and activity (right) LEDs
- 10/100/1000 BaseT RJ-45 networking link (left) and activity (right) LEDs for the processor (left) and for the BMC (right)
LED codes im Dell EMC VEP4600 Installation Guide
| id/os name | id/os name |
|---|---|
| "9"/igb4 | bmc |
| id/os name | id/os name |
|---|---|
| 3/igb1 | 5/igb3 |
| 4/igb0 | 6/igb2 |
| id/os name | id/os name |
|---|---|
| 1 | ix0 |
| 2 | ix1 |
There are two more ports facing the internal backplane for 'service chaining' use cases. Those are the ones labeled "for backplane" in the overview
OS Install pfSense kann massive Probleme mit den 1g Interfaces haben (Linkerkennung)
Man kann das nicht so einfach loesen, man muss es erst per CLI umschalten und dann geht es im GUI auch weiter.
https://www.zenarmor.com/docs/guides/disabling-hardware-offloading
(Fig. 2)
| P/N | Desc | Detail | alternate P/N |
|---|---|---|---|
| TBA | Rackmounts | 2-Post version | |
| 0G46RV | Rackmounts | 4-Post version | |
| R6TM0 | WIFI BR/BLE Expansion | 1. part number is not mentioned in any manual 2. Note it has no LTE | WLE600VX is the P/N of the M.2 WiFi module(*) |
| 92K9T | Interposer Card Assembly | (rNDC Carrier) | |
| SSD 240GB M.2 SATA | Dell OEM version of Intel SSD | ||
| SSD 960GB M.2 SATA | no info | ||
| 01FDCY | SSD 1.92TB M.2 SATA | Dell OEM verdion of Micron 5300 Pro, unknown if exact part for VEP | MTFDDAV19TDS-1AW1ZABDA |
| PSU 495W | (supported original part) | ||
| PSU 495W | XXX does also work, might cause IPMI issues | ||
| PSU 495W | XXX does also work, might cause IPMI issues | ||
| 7CRC9 | FAN | 4 Fans on 4-8 Core model, 5 Fans on 16-core model |
(*) The card shows solder points for a third IC and a shielded area is laid out on the PCB right next to it, this maybe hints at a version that had a WWAN module?
In all honesty, the WiFi/BLE card has as much electronics as a laptop motherboard. It's hard to understand why it needs so much stuff.
It makes it look unlikely that one could easily design a dual M.2/quad M.2 carrier rNDC storage expansion, even if the interface bandwidth and physical space are absolutely sufficient.
The WLE6000VX chip is pased on a Qualcomm QCA9882, it's WiFi5 / 802.11AC, 2x2MIMO, 20-80MHz width. Produced by Compex in Singapore, who have a spec sheet here and datasheet here and hardware guide with pinout.
They seem to have their own OpenWRT fork:
Supported by CompexWRT with Qualcomm Atheros reference wireless drivers or
OpenWRT/LEDE with ath10k wireless drivers,
on WPJ344, WPJ558, WPJ563, WPJ564, WPQ864, and WPQ865.
There's a DC power supply option (customer install) Part number not yet discovered, installation instructions and plug types here
SR-IOV ist im BIOS per Defaukt aktiv. VT-d etc. ebenfalls, sind verfuegbar
- i210 onboard: Managementport und Chip supported es nicht
- i350 onboard: SR-IOV deaktiviert und nicht im BIOS konfiguierbar
- i7xx onboard: SR-IOV deaktiviert und nicht im BIOS konfiguierbar
- i7xx auf rNDC: SR-IOV im BIOS konfigurierbar
Dazu gibt es eine Anleitung
Link?
Bei nutzung von Passthrough wurde nur 1 device von 4 weitergereicht, die anderen waren dann "weg"
aus u.g. VMware Doku (running on KVM)
- a. Set the spoofcheck off.
# ip link set eth1 vf 0 spoofchk off - b. Set the Trusted mode on.
# ip link set dev eth1 vf 0 trust on - c. Set the VLAN, if required.
# ip link set eth1 vf 0 vlan 3500 - d. disable GRO on host
# ethtool –K <interface> gro off tx off
zu Pruefen
- Es gibt ein NIC FW Update - aber nur fuer VMware
Ist auch im VMware SD-WAN Guide angedeutet, sehr gut moeglich, dass das also helfen wuerde. https://docs.vmware.com/en/VMware-SD-WAN/4.0/vmware-sd-wan-partner-guide.pdf
Product Software/Firmware Affected Versions Remediated Versions Link Dell Networking VEP4600 16-CORE Firmware Versions prior to 22.0.9 Version 22.0.9 or later https://www.dell.com/support/home/en-ca/drivers/driversdetails?driverid=9nppg
FW Update!!! (fuer die 710)) https://dl.dell.com/FOLDER10201251M/1/vep4600-x722-fw-rn.pdf?uid=d6f44b92-fd82-4244-40bf-353f8d1f97ae&fn=vep4600-x722-fw-rn.pdf
https://dl.dell.com/FOLDER10201235M/1/X722_V6_20_Linux.tar.gz
Siehe hier auch die resolved Issues, da passt so einiges...
Proxmox Thread bzgl. i350 Passthrough
https://forum.proxmox.com/threads/nic-card-passthrough-and-essential-things-to-think-about.80805/
"Our business is operating a radio station network with nearly 40 stations.
High availability is very important, so all our vSphere systems boot from mirrored SSDs."
cool! I did some similar stuff long long ago :-) definitely a great us ecase for the box.
Btw, I found this regarding QAT on FreeBSD. Or at least I think I found it, sorry if you had sent it to me - quite useful in any case as a starter.
https://forums.freebsd.org/threads/geli-performance-between-aesni-and-qat.95321/
I'm still waiting for the interposers, but I ordered three more today.
(I will never use eBay Global Shipping Program again.)
It annoys me how much this fact holds me back.
-Yeah, yeah, I've been in the broadcasting industry for 30 years now—I've seen a lot, but it's also my hobby, so I'm lucky.
I still climb antenna towers, which is a thrilling experience every time.
-QAT would be important for HAproxy, if I'm starting from this, let's say it's 4 Gen. scalable (or 5 Gen.):
https://builders.intel.com/docs/networkbuilders/accelerating-haproxy-on-4th-gen-intel-xeon-scalable-processors-1673295293.pdf
https://builders.intel.com/docs/networkbuilders/accelerating-haproxy-with-intel-quickassist-technology-1707819665.pdf
QAT for Load Balancing
https://intel.github.io/quickassist/PG/infrastructure_load_balancing.html
FreeBSD VF drivers and somewhere here lies the truth, with a good look back:
https://community.intel.com/t5/Intel-QuickAssist-Technology/FreeBSD-VF-drivers/m-p/1706803
or that pfSense defines Xeon D, for example....XEOND_QAT_VF
found a sloppy, rNDC assembly explosion diagram...
-You've seen this before. It could solve many SFP module compatibility issues.
Unlock Intel XL710 / X710 cards for use with any SFP+:
https://github.com/bibigon812/xl710-unlocker
@SedonD I just saw there's VROC support: Intel(R) VROC with VMD Technology 5.4.0.1039
@FlorianHeigl - Hi
Yes, I've seen it too – I'm done with the HW/FW installations, all the hardware has been updated, ESXi will be up tomorrow. I can't take the noise anymore :).
(5 FAN(s) could be very noisy 4 meters away from you)
with DiagOS from 32G DD
Package version: 4.3
Packaged images:
BMC image: 2.30
BIOS image: 3.41.0.9-25
CPLD image: 0x12
Wifi CPLD image: 0x1
X722 image: 6.20
I found a relatively new Dell NIC common firmware package and it ran on the X710s (68M95) as well (nvmupdate64e).
X722 is also fine...
Everything has gone very smoothly so far, I still need to remove the X710 SFP module restrictions, but DiagOS only contains GLIBC 2.19 and need 2.3.4, so I'm now booting from an Ubuntu Live CD.
https://github.com/bibigon812/xl710-unlocker
I hope this works, because all FS_COM 1/10G Intel-compatible modules are discarded by VEP, and all SFP ports indicate incompatibility.
- I also have FTLX8574D3BCV-IT, which ethtool -m reports without any problems.
@SedonD I was just making a video
this is for UFW on Ubuntu 20:
echo "deb http://archive.ubuntu.com/ubuntu/ focal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ focal-backports main restricted universe multiverse
deb http://archive.canonical.com/ubuntu focal partner
deb-src http://archive.canonical.com/ubuntu focal partner" > /etc/apt/sources.list
# non-interactive missing...
apt dist-upgrade -y
mkdir /install
mv /mnt/context/vep4600_ufw_4.3.zip /install
cd /install
unzip vep4600_ufw_4.3.zip
chmod 700 vep4600_ufw_4.3
./vep4600_ufw_4.3 --accept --keep --nox11 --nodiskspace --target /install/ufw
cd /install/ufw
export BMC_ADMIN=admin
export BMC_PASSWD='SERVICETAB!'
./start.sh 192.168.IPAAAAAAAADDRRRRR
grrr, ok it's failing with ufw over ipmi lan again... I only got that to work once I think. But this time I can try a few more rounds.
this is for UFW on Ubuntu 20:
I did everything with a console cable, I didn't use ipmitool.
I will only use BMC once it is installed in its final rack.
DiagOS is getting a little old:
Debian GNU/Linux 8 dellemc-diag-os
Linux dellemc-diag-os 4.9.30 #1 SMP PREEMPT Wed Jun 16 19:13:13 PDT 2021 x86_64
That's why I can't run the X710 unlocker, - - - - - -
root@dellemc-diag-os:/mnt/usb/x710_unlocker# ./xl710_unlock
./xl710_unlock: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./xl710_unlock)
root@dellemc-diag-os:~# ldd --version
ldd (Debian GLIBC 2.19-18+deb8u10) 2.19
but I'll take care of that tomorrow, I'm done for today
YafuFlash 'solution'
https://www.reddit.com/r/lowlevel/comments/9qpbpn/comment/nk1928m/
The YafuFlash in Dell UFW 4.3 is broken - might have been OK in 4.1, because I remembeR I used to use this.
Workaround is to obtain a relacement for YafuFlash
root@localhost:/install/ufw/firmware_updater# /root/Yafuflash -i -nw -ip zzzz -u admin -p admin -pnet -d 2 -mse 1 ./VEP4600-BIOS-X722-620-3.41.0.9-25.BIN
INFO: Yafu INI Configuration File not found... Default options will not be applied...
Creating IPMI session via network with address 192.168.86.12...Done
+-----------------------------------------------------------+
| YAFUFlash - Firmware Upgrade Utility v7.01.0097 |
| Copyright (c) 2020 American Megatrends International, LLC |
+-----------------------------------------------------------+
==================================================================
BIOS Firmware Details
==================================================================
Description RomImage ExistingImage
------------------------------------------------------------------
BIOS Version 0ACJF046 e?
==================================================================
Beginning BIOS Update...
Please type (Y/y) to Update or (N/n) to Cancel
Enter your Option : y
Uploading Image : 100%... done
Flashing Firmware Image : 100%... done
Verifying Firmware Image : 100%... done
Resetting the firmware..........
- mfsbsd installer (14.2 special ed.) boots with automatic serial console setup.
- used
zfsinstall -d /dev/ada0 -u /cdrom/14.2-RELEASE-amd64 -s 8G
(i don't got two SSDs) - post install set console to serial at loader prompt (press '5' twice), does not persist, need to set it in loader.conf.d later.
- for me setting this via loader.conf also disabled input at loader. there must be a better combination of settings.
- freebsd is so damn fast on this box it makes any newer one pointless :-)
QAT is not implemented in OpenSSL 3... (not compiled in, i'm checking)
Relnotes Intel Doc ID 621446 - https://www.scribd.com/document/624759333/621446-012-Intel-Qat-Softwareforfreebsd-Release-Notes
building https://downloadmirror.intel.com/733154/README.txt
from what I understand OpenSSL 3 intentionally destroyed most of the interface QAT "engine" used to hook into in OpenSSL 1.x. Something with 'providers' being favored over 'engines'. I tried to get more insights but the tl;dr was the usual "our software will be more pure if we break it" and intel didn't care to make the hardware usable either. looks not too good. FreeBSD 13 seems to be the sweet spot where it worked well.
Newer FreeBSD with OpenSSL 1.1.1 might also be easy. I'll keep following that path for a while to see / find out more. all hope is not lost, but OpenSSL 3 seems to be in a conflict situation. Intel seems to just drop their bits in the right place in this doc, https://www.intel.com/content/www/us/en/developer/articles/guide/building-software-acceleration-features-in-the-intel-qat-engine-for-openssl.html see "If you are using the distribution-provided OpenSSL" (note this is mostly QAT 2.0 stuff. But as far as I can tell it's not proven it can't work if done right.
i'm having another i-love-freebsd moment irrespective of all that. just so much nicer for me to work on and I do it far too little.
Workaround is to obtain a relacement for YafuFlash
Nice catch, I had 6.40 prepared, but then v.7.01, which is fine according to you...
- freebsd is so damn fast on this box it makes any newer one pointless :-)
Do you have time to try out a FreeBSD stable/15 branch?
@FlorianHeigl ---- (i don't got two SSDs) ----
I will examine this. The test configuration includes a Micron entp. SSD and a commercial Kingston.
@FlorianHeigl
"i'm having another i-love-freebsd moment irrespective of all that. just so much nicer for me to work on and I do it far too little."
I feel exactly the same way—FreeBSD does what needs to be done, and very quickly.
https://calomel.org/network_performance.html
https://calomel.org/freebsd_network_tuning.html
----------------------For those who want to use two rNDCs----------------------
For physical installation, you will need a M3/11mm spacer for the MC1.
M3 11mm - from Mouser - https://eu.mouser.com/ProductDetail/Wurth-Elektronik/971110321?qs=wr8lucFkNMXVmF%252BfZUxfsQ%3D%3D
(the picture also shows a plastic type, but do NOT use it, because metal is required for PCB(s) grounding transfer.)
for WIFI/BT card, has a longer one factory-installed
I'm trying an upgrade into 15.0-BETA1.
Then I'd like to see something, the using QAT, maybe via kTLS. No matter how simple ;-)
I've seen the default power management settings don't work with the OS (FreeBSD doesn't find any hooks for it), so I'll probably put my last energy into going through the BIOS instead. (I get 70W measured power draw by the PSU and 35W via IPMI. I think the culprit is the fan speeds.
Not sure what's feasible to work on after that ^^
I wanted to look in the SNMP MIB to see if there's some dieing gasp trap support, that would be fun to have on a "server'.
The main success for me is knowing I'll now be able to update all installed BIOS/BMC remotely. And I'm still wondering why this BMC should not be affected by the redfish vuln. Got no reply from Dell when I wrote to them but there's no announcement either. Maybe Dell had also stripped out some of the Redfish bits.
FreeBSD 15 works:
Last login: Sat Oct 18 18:42:51 2025 from 192.168.10.151
FreeBSD 15.0-BETA1 (GENERIC) releng/15.0-n280655-36a923a476dc
booted :>
I found a relatively new Dell NIC common firmware package and it ran on the X710s (68M95) as well (nvmupdate64e).
could you share the link for that if you still got it? I feel my burnout when going through the support pages, I looked for version 9.84 but didn't get lucky.
googled, there's a FreeBSD version of the tool, I'll be able to give it a shot.
This thing can take DC PSUs - it would be kinda funny to run it off PoE++
Oh, and btw, I only was on a radar tower once, got seasick once I was up, couldn't work at all, just curled up on the floor against a wall till we were done and someone could help me down. I think I peeked into the IT shelter for a moment but it was darn chaotic and, well, the view into the horizon was definitely prettier.
could you share the link for that if you still got it?
I was lazy over the weekend, I didn't do anything. I don't usually bring work home, but it's here now and fan(s) is loud, so I didn't want to disturb my family.This also helped me decide on a lazy weekend.
Intel NIC Family Version 24.0.0 firmware XX710, XXV710 és XL710 (release 21 Jul 2025):
https://www.dell.com/support/home/en-us/drivers/driversdetails?driverid=dc20j&lwp=rt
I extracted the RHEL package under Ubuntu.
chmod +x ./Network_Firmware_DC20J_LN_24.0.5_A00.BIN
sudo ./Network_Firmware_DC20J_LN_24.0.5_A00.BIN --extract ./firmware
& ./nvmupdate64e
and both X710s in the rNDCs are perfectly updated via console.
Yeah, it can often be exhausting to spend days searching the internet to install/learn a new tool, that's true, so I search and take notes in advance whenever I can. It has its own flavor.
I've seen the default power management settings don't work with the OS (FreeBSD doesn't find any hooks for it), so I'll probably put my last energy into going through the BIOS instead. (I get 70W measured power draw by the PSU and 35W via IPMI. I think the culprit is the fan speeds.
Not sure what's feasible to work on after that ^^
I wanted to look in the SNMP MIB to see if there's some dieing gasp trap support, that would be fun to have on a "server'.
- The fans definitely have an impact to this. I've noticed that under DiagOS, with the original BIOS (the one I received: 3.41.0.9-18), the PSU consumption was 75-80W and the fans were spinning like crazy.
They did not recover after the psutool, fantool or edatool tests; all FAN(s) stuck on 25K(?) rpm and making loud noises, and only a reboot brought them back to a tolerable noise level. From this, I conclude that the code for this FAN control is not really adequate or mature.
In the latest BIOS, can tell that they have been working on this, but still don't symphatetic...
Edit (21-10-2025): I just found this, so apparently others have noticed it too.
https://forums.servethehome.com/index.php?threads/dell-emc-vep-4600-high-fan-speed.41880/
I think it's unnecessary to load five such small fans in this way and pollute the environment with noise.
The noise is seriously comparable to that of our Cisco UCS-240M7 servers.
This is the next thing I will work on, because in an air-conditioned server room, it is unnecessary to maintain a base speed of 12K, as all MOBO sensors show values below 28 degrees.
-
I've already looked at SNMP also, but that will have to wait for another day. It won't be easy, as it's an Intel "aolnprxy" based MIB.
Maybe I'll work on this in the winter, because I want to build it under LibreNMS and I don't see anything similar yet.
It's not easy to integrate a new "OS" under LibreNMS, which is what they call new devices, but you may already be familiar with LibreNMS. -
Is this what you meant, CVE-2024-54085? - We have a couple of TrueNAS units with Supermicro motherboards that have already been updated.
You saved me time with the FreBSD 15 test, so I'll set up one of the VEPs this way as a base. THX
Oh, and btw, I only was on a radar tower once, got seasick once I was up, couldn't work at all, just curled up on the floor against a wall till we were done and someone could help me down. I think I peeked into the IT shelter for a moment but it was darn chaotic and, well, the view into the horizon was definitely prettier.
UBNT AirFiber link now 2025 September, I'm fine with these altitude issues. :)
Do you know anything about this MOBO slot?
"
Linux
SR-IOV Settings
i210 onboard: Managementport und Chip supported es nicht
i350 onboard: SR-IOV deaktiviert und nicht im BIOS konfiguierbar
i7xx onboard: SR-IOV deaktiviert und nicht im BIOS konfiguierbar
i7xx auf rNDC: SR-IOV im BIOS konfigurierbar
Dazu gibt es eine Anleitung
Link?
PCI Passthrough
Bei nutzung von Passthrough wurde nur 1 device von 4 weitergereicht, die anderen waren dann "weg"
"
-----ESXi 8 custom Dell U3-----
Under ESXi 8, all interfaces work almost without any problems, using SR-IOV and passthrough, except MC1 rNDC - NO SR-IOV (NOT CAPABLE?????)
(I couldn't find a separate setting for this in the BIOS; is it designed specifically for Wi-Fi/Bluetooth? - maybe that's why there isn't)
Intel I350 / I210:
Intel X710 in rNDC (MC2)
Intel X710 in rNDC (MC1)
Intel X722 onboard
-----------In the end, this worked too.-----------------
Unlock Intel XL710 / X710 cards for use with any SFP+:
https://github.com/bibigon812/xl710-unlocker
Under Ubuntu, both 68M95s has been unlocked, and any SFPs will be accepted after that.
Thank you for putting this information together for the VEP4600.
Can anyone confirm the same behavior I'm seeing on the CPLD firmware. I've attempted to flash V 1.2, always showing flashing was successful. After a power cycle, the CPLD version number always reports as 1.0 on the console.
The VEP4600 started yesterday with BMC v1.20 firmware. After the upgrade to BMC v2.30 I noticed 4 virtual AMI CDROM devices connected to USB.
root@dellemc-diag-os:/# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr2 11:2 1 1024M 0 rom
sr0 11:0 1 1024M 0 rom
sda 8:0 0 924.1G 0 disk
|-sda2 8:2 0 1G 0 part /
-sda1 8:1 0 128M 0 part
sr3 11:3 1 1024M 0 rom
sr1 11:1 1 1024M 0 rom
root@dellemc-diag-os:/# dmesg
...
...
...
[ 4.360238] usb 1-3: new high-speed USB device number 2 using xhci_hcd
[ 4.530303] hub 1-3:1.0: USB hub found
[ 4.530484] hub 1-3:1.0: 5 ports detected
[ 4.814232] usb 1-3.1: new high-speed USB device number 3 using xhci_hcd
[ 4.901739] usb-storage 1-3.1:1.0: USB Mass Storage device detected
[ 4.901839] scsi host5: usb-storage 1-3.1:1.0
[ 5.957938] scsi 5:0:0:0: CD-ROM AMI Virtual CDROM0 1.00 PQ: 0 ANSI: 0 CCS
[ 5.959361] sr 5:0:0:0: [sr0] scsi-1 drive
[ 5.959362] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 5.959486] sr 5:0:0:0: Attached scsi CD-ROM sr0
[ 5.959563] sr 5:0:0:0: Attached scsi generic sg1 type 5
[ 5.959992] scsi 5:0:0:1: CD-ROM AMI Virtual CDROM1 1.00 PQ: 0 ANSI: 0 CCS
[ 5.961654] sr 5:0:0:1: [sr1] scsi-1 drive
[ 5.961768] sr 5:0:0:1: Attached scsi CD-ROM sr1
[ 5.961857] sr 5:0:0:1: Attached scsi generic sg2 type 5
[ 5.962849] scsi 5:0:0:2: CD-ROM AMI Virtual CDROM2 1.00 PQ: 0 ANSI: 0 CCS
[ 5.964385] sr 5:0:0:2: [sr2] scsi-1 drive
[ 5.964527] sr 5:0:0:2: Attached scsi CD-ROM sr2
[ 5.964596] sr 5:0:0:2: Attached scsi generic sg3 type 5
[ 5.965208] scsi 5:0:0:3: CD-ROM AMI Virtual CDROM3 1.00 PQ: 0 ANSI: 0 CCS
[ 5.966637] sr 5:0:0:3: [sr3] scsi-1 drive
[ 5.966759] sr 5:0:0:3: Attached scsi CD-ROM sr3
[ 5.966825] sr 5:0:0:3: Attached scsi generic sg4 type 5
...
...
...
root@dellemc-diag-os:/#
I haven't found any exposed API for virtual media support on the VEP4600 through the BMC or otherwise, so I assume that the virtual CDROMs are nonfunctional with this BMC (just like the redfish endpoint). I'd be very pleased to be wrong so please prove me wrong.
These virtual devices can be disabled by using the BMC serial console to edit /conf/vmedia.conf, changing num_cd=4 to num_cd=0 and run "/etc/init.d/vmscript restart". After that, the phantom CD-ROM devices are gone.
root@dellemc-diag-os:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 924.1G 0 disk
|-sda2 8:2 0 1G 0 part /
-sda1 8:1 0 128M 0 part
root@dellemc-diag-os:/#
The change to /conf/vmedia.conf appears to be persistent across BMC reboots.
Can anyone confirm the same behavior I'm seeing on the CPLD firmware. I've attempted to flash V 1.2, always showing flashing was successful. After a power cycle, the CPLD version number always reports as 1.0 on the console.
-Yes, I can confirm this as behavior, since the normal (short)power cycle is not always sufficient for some reason.
I may have read somewhere that the VEP should be completely shut down for several minutes.
Especially if you used point (1) for updates.
"1. Automatically update all firmware components"
After the upgrade to BMC v2.30 I noticed 4 virtual AMI CDROM devices connected to USB.
These vCDs do appear after 2.30, and they don't bother me for now, but it's on my to-do list to find out what they can be used for.
Since documentation for this VEP device(s) is scarce, there is still much to discover about them, but they work perfectly as network appliances in any env.s.
My last tests were with FreeBSD15, where QAT is not functional.
1.x devices are not supported even outside the tree.
However, pfSense PLUS works well and QAT is also immediately available, as they have their own implementation for this.
-Ubuntu Server is totally fine, no problems, Debian too.
-ESXi8 U3 is problem-free, although ESXi should only worth installing on the 16C32T version.
-VyOS too
It has a couple of quirks, but if you overlook those, it's a really great "iron" for a network environment.
my assumption is that you have to pull the plug after CPLD update or that it depends on the order of how you update things.
My last one had it already at right level, so I don't know for sure.
Generally I consider this a pull the plug kind of thing.
@SedonD that's crap news with regard to FreeBSD 15 and QAT 1.0. I was kinda expecting to end up in that situation when I did the upgrade test ;-)
I've given some 8955 offload cards to one of the OPNsense devs earlier in the year, so maybe there will also be a chance. I have QAT 1.0 on my list for compression offload in ZFS mostly, but I know it's a race against time. Especially since it's happening on multiple fronts with the OpenSSL "engine" also being deprecated and left to code-rot. A bad race, and I'm not gonna run for anyone ;-)
But generally, I think it will be possible to turn the tide on this since even if devs drop some code we can slowly take it over. On Linux the situation with /dev/crypto and all async crypto functions is similar. I'd say not to worry and see how the situation evolves over time.
pull the plug after CPLD update
Yes, need a total blackout. :-) - I also experienced this during my own run.
that's crap news with regard to FreeBSD 15 and QAT 1.0.
It works in FreeBSD 13 (this is the last release where yes), so I've already set up an 8C16T model and it's been running for weeks with HAproxy (FreeBSD 13 based). It's turned out to be great stuff.
I've read a lot about this, opinions are divided, but Intel has abandoned this 1.x thing.
You might think it's because of the QAT 2.x stuff in the newer CPU. It's always business at first glance, and later only users. ):
Netgate has continued its own QAT branch into version 15, and it works well.
I'll say it quietly: you have to pay here too, because QAT is only available in the PLUS version.
Thank you for your follow-up.
-I've been waiting for three Dell interposers for two weeks, this is nonsense from the Portuguese postal service (CTT).
(I paid the customs duty more than 8 days ago, but the tracking shows that it was shipped in the wrong direction.)
That's why I haven't written anything yet. I've prepared everything down to the last detail, and I even have the console cable here in my hand. :)
-Tomorrow I will take photos of the rNDC for you or whoever needs it, as I have a factory 68M95 in my unit and two more in my possession.
-I found a document that briefly mentions FreeBSD, so I hope that won't be a problem.
-Broadcom download(s) is totally insane YES, -but ESXi 7 can still be downloaded from the Dell website, and I collected ESXi 8 EMC custom images with add-ons, so I can start from there, I hope.
-I know of 24 more VEPs (8C16T + 16G RDIMM, no ssd) in Germany, if you're interested, - they're in good condition.
(I won't write it here, because if it works for us, we'll need 10 more - but let me know in PM if you need)
-Dell support told us that the X722 cannot currently go above 6.20, but with an NVM update, the X710 can go to 6.50, and in theory, the I350 can too. (on 4600!)
-I got two 99% S.M.A.R.T Micron M.2 SSDs, and I'm going to use them.
In vCenter, I can easily add drivers (VIBs) to standard ESXi packages, and Dell has described the necessary components in a .txt file for VEP.
(at least, I hope so - "easily")
-I am currently attempting to re-register the two VEPs with Dell in order to receive support. Of course, we have to pay for this, but I need to transfer them to our company name.
We have a Broadcom account and a vSphere subscription that is still active, but I cannot access VEP downloads without a Dell support token.
(It's true that our vSphere will only be available until the end of this year.)
I found a picture from my first series.... - I always take lots of pictures: