Installing FreeBSD from an IPMI serial-over-lan console

C

Christopher

I just got 16 new servers delivered to the data center and am getting ready to rack and install them next week. The servers come with a Supermicro IPMI management system on the mainboard that does serial over LAN. I've set up the IPMI interface and can watch the machine attempt to boot from the currently empty hard drives with ipmitool. Pretty sweet.

Does anyone have any experience with installing FreeBSD (7.2 or 8.0 amd64 in this case...) from the IPMI serial-over-LAN console? Does FreeBSD treat it just like a standard serial port?
 
I've almost same hardware here ( Supermicro + Xeon + IPMI). Yes, FreeBSD treat it just like a standard serial port. Now, all you've to do is configure install server to deploy Linux or FreeBSD. Under FreeBSD 7.2 you need to add the following kernel option and recompile the stuff to avoid KVM keyboard problem.
Code: 
nodevice ums

Here is my kernel config:
Code: 
include GENERIC
ident www-167
options SMP
nodevice ugen
nodevice uhid
nodevice ulpt
nodevice umass
nodevice ums
nodevice ural
nodevice urio
nodevice uscanner
nodevice aue
nodevice axe
nodevice cdce
nodevice cue
nodevice kue
nodevice rue
options CONSPEED=19200
options QUOTA
device pf
device pflog
device pfsync
options         ALTQ
options         ALTQ_CBQ        # Class Bases Queuing (CBQ)
options         ALTQ_RED        # Random Early Detection (RED)
options         ALTQ_RIO        # RED In/Out
options         ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
options         ALTQ_PRIQ       # Priority Queuing (PRIQ)
options         ALTQ_NOPCC      # Required for SMP build
options DEVICE_POLLING
options HZ=1000
Rebuild your kernel.
 
Thanks for your information!

Quick question, if one doesn't recompile the kernel (trying to keep it simple for freebsd-update(8)) what will the usb mouse driver do?
 
Christopher said:
Thanks for your information!

Quick question, if one doesn't recompile the kernel (trying to keep it simple for freebsd-update(8)) what will the usb mouse driver do?
Click to expand...

The integrated KVM appears as a single USB device for the virtual floppy/cdrom, a keyboard and a mouse. The issue appears to be in FreeBSD's ums driver ums. When the FreeBSD kernel attempts to attach and setup the KVM mouse, it fails. This, in turn, causes the kernel to stop attaching probing and drivers to the device. In other words if you compile kernel, mouse will never work but keyboard will and you will able to fix the issue remotely or boot FreeBSD in a single user mode with the following settings at a boot prompt over KVM itself:
Code: 
load usb.ko
set boot_serial=NO
set boot_multicons=NO
set console=vidconsole
boot -s

I've not tested FreeBSD 8 as all servers are in production. I will wait for 8.1 or 8.2 before we start migration.
 
Installation story so far

In case anyone is interested, I'll put some notes on what I've discovered so far.

The IPMI serial-over-lan (SOL, that acronym is apt) serial port is BIOS wired to COM3 (sio2), I/O port 0x3E8, IRQ 5. I've managed to coax the PXE Loader and kernel boot to talk to it by recompiling it on another machine with BOOT_COMCONSOLE_PORT=0x3E8 as a make argument. However, I've not gotten sysinstall to play with it. It seems to always default to COM1, regardless of any tweaks to device.hints. If I hook up a null modem to the serial port, I do indeed get sysinstall interaction, so its at least realized we are trying a serial install.

The KVM over LAN interface (via IPMIView or the Java applet from the impeded web server on the BMC) seems to work.. occasionally... with the GENERIC, unmodified kernel. I haven't nailed down a pattern yet, but sometimes the keyboard works, sometimes it doesn't. It seems to help if one nudges it first with a physically attached PS/2 keyboard.

The x86 world is getting closer to having truly remotely managed systems at least.
 
Ditto. I've never managed to get FreeBSD installed over SOL. Likewise on this morning's messing about with an 8.0 install which can't mount root I've not been able to get GENERIC to use the second serial port (it's COM2 on Dell hardware, not COM3)

Be glad you've got a KVM over IPMI. Dell users seem to be stuck with SOL.

At least it's better than serial cables, serial terminal servers and remote power bars which someone always mucks up the port labelling on ;)
 
You must log in or register to reply here.
Back
Top