38

The SuperMicro X8SIE-F board has two dedicated LAN interfaces for the operating system (LAN1/2) and one dedicated LAN interface for IPMI.

Is it possible to configure IPMI to use one of the LAN1/2 interfaces, instead of the IPMI port? If so, what is the procedure?

7 Answers 7

61
+100

Jiri's on the right track with the three options (Dedicated, Share, Failover) for the IPMI interface. The short answer is that yes, you can use LAN1 instead of the dedicated IPMI port, and it generally works that way with the default BIOS settings. It's not possible to run the IPMI on the LAN2 interface.

Here's a more detailed description of the three options:

  • Dedicated: Always use the dedicated IPMI interface. This is the option you want if you're trying to have the simplest setup, at the expense of additional cabling.

  • Shared: Always use the LAN1 interface. This is the option you want if you're trying to reduce your cabling to each server, and understand the tradeoffs. Under the covers, there's a virtual switch in hardware that's splitting out traffic to the IPMI card from traffic to the rest of the system; the IPMI card has a separate MAC address to differentiate the traffic. On modern Supermicro boards, you can also set the IPMI traffic to run on a different VLAN from the rest of the system, so you can tag the IPMI traffic. There are some definite security implication to this design; it's not difficult for the main system to access the IPMI network, if you were trying to keep them separated. A failure of the LAN1 interface often means that you lose primary and out-of-band connectivity at the same time.

  • Failover (factory default): On boot, detect if the dedicated IPMI interface is connected. If so, use the dedicated interface, otherwise fall back to the shared LAN1. I've never found a good use for this option. As best I can tell, this setup is fundamentally flawed - I haven't tested it extensively, but I've heard reports it'll fail to detect the dedicated interface in many circumstances because the upstream switch isn't passing traffic - for example, after a power outage if the switch and system come up simultaneously, or if the switch is still blocking during the spanning tree detection. Combine this with the fact that the check only happens at boot, and it's just generally hard to control what interface you end up using.

8
  • 11
    Failover results in more problems than it's worth, in my opinion. There's no easy way to tell if your IPMI NIC has failed over or not. It requires a power cycle (not just a reboot) to reset the port back to the dedicated one. Feb 21, 2012 at 0:13
  • Thanks a bunch for that detailed explanation. In IPMIView I see the three options, but there's no "Help" button or explanation to describe what they do.
    – 80skeys
    Feb 21, 2012 at 19:04
  • 2
    On linux, ipmitool mc reset cold mostly helps in that situation, but this can also cause strange behaviour on the host for several seconds since the kernel has to deal with the devices the BMC emulates behaving like they suddenly were ripped off the bus. Another problem with failover is that a) it is the default, b) it can unexpectedly make a dedicated sideband lan lose connectivity, c) throw the bmc straight in harms way into a DMZ. May 6, 2012 at 16:23
  • 3
    note that the factory default is IPMI failover (at least on the 5 new SuperMicro based servers I just bought) which leads to a lot of.. confusion. This is dangerous! Oct 23, 2012 at 19:23
  • 2
    @StefanLasiewski: See this. Aug 19, 2013 at 21:54
39

The following raw commands will surely prove useful to the next person who reads this topic. I got these directly from Supermicro support. Cheers:

To get LAN mode: ipmitool raw 0x30 0x70 0x0c 0.

To set LAN mode dedicated: ipmitool raw 0x30 0x70 0x0c 1 0.

To set LAN mode onboard/shared: ipmitool raw 0x30 0x70 0x0c 1 1.

To set LAN mode failover: ipmitool raw 0x30 0x70 0x0c 1 2.

These raw values will also work with ipmicfg (just remember to use -raw instead of raw).

After setting the required LAN mode, do not forget to do a hard reboot.

5
  • 5
    i came here again, i would love to upvote again... Dec 15, 2017 at 3:11
  • Your comment makes me happy
    – lobi
    Jul 19, 2018 at 2:55
  • 1
    I've hacked a bit around it gist.github.com/FlorianHeigl/4d90261ceabd056ebb6ab2ca6aa814ac Jul 20, 2018 at 9:25
  • 5
    I had a hard time finding what the results of the get command meant, so I thought I would post them here for anyone else: 0x00 = Dedicated, 0x01 = Onboard / Shared, 0x02 = Failover
    – Dominic P
    Aug 30, 2018 at 5:10
  • Changing from Failover to Shared worked for me right away with no need for a reboot. Aug 29, 2019 at 10:06
15

In these boards, the ipmi interface can only be configured on the 1rst LAN or in the dedicated one, and you can choose it by issuing some raw commands. My steps to enable the dedicated LAN on a X8DTU-F board are these:

#install ipmitool (this is for debian)
apt-get install ipmitool
#insert the kernel modules needed for ipmi
modprobe ipmi_devintf
modprobe ipmi_si
modprobe ipmi_msghandler
#get the current mode (01 00 is dedicated mode)
ipmitool raw 0x30 0x70 0x0c 0
#send the raw command to enable dedicated lan
ipmitool raw  0x30 0x70 0xc 1 1 0

You can now use ipmitool to assign an ipaddress/netmask to the interface, and start working with it normally.

If you want to know all the comands in raw mode for the supermicro ipmi, there's a list here

1
  • 1
    Most useful answer here, especially the link to the SMC FAQ, which makes it clear that different motherboards use slightly different versions of this command.
    – Nemo
    Oct 11, 2017 at 0:16
14

I figured it out. The way it works is pretty hokey, but here it is:

It depends on whether there is a network cable plugged into the dedicated IPMI port, before the server receives power. If a cable is present, then IPMI is automatically run on that port. If there is no cable in that port, then IPMI is automatically assigned to the LAN1 port (shared with the operating system).

These settings are not persistent through power cycles unless you make them permanent using IPMI View tool > BMC Settings. If you do not do this, then every time the server gets power cycled, it will test the cable again and reassign IPMI accordingly.

1
  • 2
    Mark yourself as having the right answer, this could be quite useful for other folks
    – mfinni
    Oct 23, 2012 at 19:44
5

You can also use ipmitool to set the lanport:

sudo ipmiutil smcoem lanport dedicated

Then check that your config is right:

sudo ipmiutil lan -e

You should see the line:

SuperMicro Lan Interface  :  00     :  Dedicated

Cheers!

2

I have X8SIL-F-B and if you connect to BMC via lan with browser, you can see in 'Configuration - Network' that for LAN interface there are more options (dedicate, share, failover).

But I haven't tested it.

1

Here is another usefull Supermicro IPMI tool for users having issues: IPMICFG This has helped me out of a few jams before.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .