Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I would like to install bonding with 4 links with mode 4. but only "download/receiving" works with bondig. for transmitting the system chooses one link.

ifconfig

bond0     Link encap:Ethernet  HWaddr 90:E2:BA:0F:76:B4
          inet addr:ip  Bcast:ip  Mask:255.255.255.248
          inet6 addr: fe80::92e2:baff:fe0f:76b4/64 Scope:Link
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:239187413 errors:0 dropped:10944 overruns:0 frame:0
          TX packets:536902370 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:14688536197 (13.6 GiB)  TX bytes:799521192901 (744.6 GiB)

eth2      Link encap:Ethernet  HWaddr 90:E2:BA:0F:76:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:54969488 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2537 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3374778591 (3.1 GiB)  TX bytes:314290 (306.9 KiB)

eth3      Link encap:Ethernet  HWaddr 90:E2:BA:0F:76:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:64935805 errors:0 dropped:1 overruns:0 frame:0
          TX packets:2532 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3993499746 (3.7 GiB)  TX bytes:313968 (306.6 KiB)

eth4      Link encap:Ethernet  HWaddr 90:E2:BA:0F:76:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:57352105 errors:0 dropped:2 overruns:0 frame:0
          TX packets:536894778 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3524236530 (3.2 GiB)  TX bytes:799520265627 (744.6 GiB)

eth5      Link encap:Ethernet  HWaddr 90:E2:BA:0F:76:B4
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:61930025 errors:0 dropped:3 overruns:0 frame:0
          TX packets:2540 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:3796021948 (3.5 GiB)  TX bytes:314274 (306.9 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:62 errors:0 dropped:0 overruns:0 frame:0
          TX packets:62 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:5320 (5.1 KiB)  TX bytes:5320 (5.1 KiB)

those are my configs:

DEVICE="eth2"
BOOTPROTO="none"
MASTER=bond0
SLAVE=yes
USERCTL=no
NM_CONTROLLED="no"
ONBOOT="yes"


DEVICE="eth3"
BOOTPROTO="none"
MASTER=bond0
SLAVE=yes
USERCTL=no
NM_CONTROLLED="no"
ONBOOT="yes"



DEVICE="eth4"
BOOTPROTO="none"
MASTER=bond0
SLAVE=yes
USERCTL=no
NM_CONTROLLED="no"
ONBOOT="yes"


DEVICE="eth5"
BOOTPROTO="none"
MASTER=bond0
SLAVE=yes
USERCTL=no
NM_CONTROLLED="no"
ONBOOT="yes"

DEVICE=bond0
IPADDR=<ip>
BROADCAST=<ip>
NETWORK=<ip>
GATEWAY=<ip>
NETMASK=<ip>
USERCTL=no
BOOTPROTO=none
ONBOOT=yes
NM_CONTROLLED=no

cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: slow
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 4
        Actor Key: 17
        Partner Key: 11
        Partner Mac Address: 00:24:51:12:63:00

Slave Interface: eth2
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:76:b4
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth3
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:76:b5
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth4
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:76:b6
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth5
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:0f:76:b7
Aggregator ID: 1
Slave queue ID: 0

/etc/modprobe.d/bonding.conf

alias bond0 bonding
options bond0 mode=4 miimon=100 updelay=200
#downdelay=200 xmit_hash_policy=layer3+4 lacp_rate=1

Linux:

Linux 3.0.0+ #1 SMP Fri Oct 26 07:55:47 EEST 2012 x86_64 x86_64 x86_64 GNU/Linux

what i've tried:

downdelay=200 xmit_hash_policy=layer3+4 lacp_rate=1

mode 6
share|improve this question
    
What are you connecting the server to? Vendor? Configuration? –  rnxrx Oct 26 '12 at 3:20
    
what you mean by vendor? it's conected to a switch in Data center –  Crazy_Bash Oct 26 '12 at 11:36
    
OK, what make and model is the switch, then? –  Tom O'Connor Oct 26 '12 at 11:45
    
cat /proc/net/bonding/bond0 would also be useful. –  chutz Oct 27 '12 at 7:01

3 Answers 3

up vote 0 down vote accepted

Before you start debugging the bonding configuration, have you verified that each network link can work well on its own? If nothing works when you bring eth4 down (ifdown eth4) then there is most likely a problem with your wiring.

Also, you should probably try some of the more simple bonding modes first. Try balance-rr (mode=0) and see if you can see traffic on all interfaces. Also look in /proc/net/bonding/bond0 to verify the current status of your configuration.

share|improve this answer
    
It's a bit of a risk for me, becouse if i make one mistake and the server goes offline i would need to drive all the way to my DC. i found out that my bonding actually works only on receiving, and only one of them works on transmitting. –  Crazy_Bash Oct 27 '12 at 6:55
    
In that case it is probably what I suspected. The outgoing link selection always picks the same link when you connect to the same IP. Try pinging other addresses in the same subnet and you may see packets on the other links. –  chutz Oct 27 '12 at 6:59
    
Thanks! your answer lead me to the solution. instead of putting bonding options only in /etc/modprobe.d/bonding.conf i put them also in /etc/sysconfig/network-scripts/ifcfg-bond0 and now it's working –  Crazy_Bash Oct 27 '12 at 9:07

You should read this document: Linux Bonding

share|improve this answer

http://wiki.mikrotik.com/wiki/Manual:Interface/Bonding says:

LACP balances outgoing traffic across the active ports based on hashed protocol header information and accepts incoming traffic from any active port. [...] How hash is calculated depends on transmit-hash-policy parameter.

transmit-hash-policy: layer-2 - Uses XOR of hardware MAC addresses to generate the hash. This algorithm will place all traffic to a particular network peer on the same slave.

So, if you transmitting from bond0 only to a single destination in the network, that is to say, to a singe MAC address, all the frames go out from only one of your ports. This is an expected behavior. If you'd transfer to some other MAC, chances are that the hash algorithm would select one of 3 other ports from your 4 ports (chance is 75% to be exact). This is to ensure that order of delivery of frames would be the same as sending order.

PS: If your transfer goes through the IP gateway, this counts as a single-MAC transfer, too. Quite a common scenario.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.